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Abstract 

Undersea  technology  is  on  the  verge  of  equipping  remotely  operated  vehicle  (ROV)  pilots 
with  a  three-dimensional  (3-D),  real-time  display  incorporating  data  from  a  wide  variety  of 
sensors  including  sonar  (sound  navigation  and  ranging),  cameras,  and  lasers.  Effective 
collection,  computation,  and  presentation  of  this  data  to  the  pilot  in  a  single  display  presents 
hardware,  software  and  human  factors  problems.  This  thesis  focuses  on  human  factors 
issues  associated  with  the  display  of  information  which  could  enhance  the  pilot's  efficiency 
of  performance.  Background  information  on  human  factors  engineering,  3-D  computer 
graphics  displays,  and  application  of  the  3-D  perspective  display  precede  the  details  of  the 
experiment. 

Five  specific  display  enhancements  tested  include  altering  the  displayed  field  of  view, 
providing  a  screen  grid,  displaying  the  current  range  to  the  target  of  interest,  using  a 
vertical  color  scheme,  and  controlling  the  display  update  rate.  Seven  tests  measure  the 
effects  of  these  display  enhancements  on  the  simulated  piloting  of  an  ROV.  The  effects  of 
the  ROV  simulation  and  operator  learning  curves  are  removed  to  compare  performance 
changes  due  to  the  various  enhancements  directly.  Operator  comments  during  and  after 
testing  as  well  as  test  monitor/author  observations  provide  insight  into  the  experiment  Test 
result  implications  for  system  design  trade-offs  are  discussed  in  detail.  Recommendations 
for  future  research  and  the  proposed  construction  of  a  fully  equipped  ROV  simulator 
complete  the  work. 
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Chapter  1 
Introduction 


1.1     Background 

Sonar  (Sound  Navigation  and  Ranging)  is  a  technique  for  underwater  range  finding 
that  has  existed  for  more  than  fifty  years.  The  author's  thirteen  years  of  experience  with 
military  and  commercial  sonars  indicate  that  the  techniques  employed  to  present  sonar 
information  to  the  user  significantly  lag  the  technology  used  in  areas  such  as  hardware 
design,  beam  forming,  and  data  processing,  and  unnecessarily  limit  the  bandwidth  of 
information  transfer.  The  wide  variety  of  display  designs  and  information  formats  proves 
confusing  to  all  but  the  most  adept  and  experienced  users.  Additionally,  the  typical 
restriction  of  the  presentation  to  two  coding  dimensions,  coincident  with  the  physical 
dimensions  of  the  cathode  ray  tube  (CRT),  severely  restricts  the  ability  to  effectively 
convey  large  amounts  of  data. 

The  Woods  Hole  Oceanographic  Institution's  (WHOI)  Deep  Submergence  Labora- 
tory (DSL)  operates  several  seagoing  craft  including  remotely  operated  vehicles  (ROVs), 
whose  pilot  "flies"  the  deep-diving  ROV  from  an  operations  center  located  on  a  mother  ship 
on  the  surface  above.  Within  video  camera  range  the  pilot's  display  resembles  the  view  a 
driver  sees  out  the  front  windshield  of  an  automobile,  with  some  loss  of  depth  perception. 
This  video  display  format  is  "natural"  for  the  pilot  and  provides  a  large  information  transfer 
bandwidth.  Suspended  sediment  near  the  bottom  frequently  limits  the  already  restricted 
visual  range  of  camera  equipment  to  a  couple  of  meters.  Beyond  visual  range  the  pilot 
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must  rely  on  the  customary  two-dimensional  (2-D)  active  (transmit  and  receive)  sonar 
presentations. 

Technological  advances  currently  provide  the  capability  to  combine  undersea  data 
from  a  number  of  sensors  including  active  sonars,  video  tape  and  electronic  still  cameras, 
manipulators,  and  laser  range  finders.  These  data  can  be  assimilated  and  processed  in  real 
time  to  provide  large  quantities  of  high  quality  information.  The  data  generated  can  be  used 
to  provide  the  ROV  pilot  with  a  real-time,  three-dimensional  (3-D),  perspective  display 
similar  to  that  of  the  video  camera  when  beyond  visual  range.  This  more  natural  view 
would  ease  the  pilots  workload  and  improve  the  efficiency  of  work  conducted  beyond 
visual  camera  range.  Such  a  perspective-view  display  is  expected  to  be  dominated  in  the 
foreground  by  high-resolution  video  data,  gradually  transitioning  to  dominance  by  other 
sensors  as  the  limits  of  video  range  are  encountered.  The  envisioned  3-D  display  must  be 
carefully  designed  to  best  improve  the  ROV  pilot's  efficiency. 

1.2     Research  Objectives 

A  variety  of  display  enhancements  have  been  used  in  military  and  commercial 
sonars.  This  thesis  looks  at  the  individual  effects  of  six  specific  display  enhancements: 
widening  the  field  of  view  (FOV)  to  provide  more  information  to  the  pilot,  2-D  and  3-D 
gridding  for  improved  angular  and  depth  perception,  vertical  color  schemes  for  better 
altitude  versus  depth  perception,  direct  range  readout  to  shorten  the  search  and  location 
cycle,  and  shortening  the  display  update  rate.  Seven  experiments  to  isolate  and  quantify  the 
individual  effects  of  these  enhancements  are  described  in  detail.  The  analysis  allows  direct 
comparison  of  the  effectiveness  of  the  six  enhancements. 
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1.3     Terminology 

Terminology  is  critical  to  comprehension.  Several  important  definitions  include: 

•  altitude  -  the  distance  above  the  floor  of  the  ocean. 

•  data  -  "raw"  numbers  from  which  information  can  be  extracted. 

'  •  data  point  -  one  of  a  set  of  25  x,  y,  and  z  coordinates  chosen  for  a  target  course. 

•  depth  -  the  perceived  distance  into  a  video  or  computer  display.  This  is  a  spatial 

perception  of  range. 

•  display  -  the  computer  screen  dedicated  to  graphics  information. 

•  driving  -  the  act  of  operating  the  simulated  ROV  for  testing. 

•  heading  -  the  compass  direction  in  which  the  vehicle  is  pointed. 

•  monitor  -  the  computer  screen  dedicated  to  printed  information,  including  update 

indication,"target  hit"  and  "mission  complete"  indication,  target  range 
indication  (test  6),  and  vehicle  world-coordinate  position,  heading,  and 
velocities.  This  term  is  also  used  to  describe  the  actions  of  the  author  as 
the  testing  supervisor. 

•  operator  -  a  test  subject  who  drove  the  simulated  ROV. 

•  pilot  -  a  qualified  flyer  of  operational  ROVs. 

•  piloting  -  the  act  of  flying  a  operational  ROV. 

•  target  -  the  numbered,  cubic  wireframe,  data  point  visualizing  tool. 

•  target  course  -  one  of  19  (lettered  A-R,  X)  specific  sequences  of  data  points  and 

associated  targets. 

•  update  -  the  complete  refreshing  of  a  computer  screen. 

•  vertical  -  the  direction  opposite  to  that  in  which  gravity  acts.  A  vertical  color 

scheme  encodes  altitude  information  using  a  color  scale  or  look-up  table. 
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1.4     Overview 

Chapter  2  provides  background  information  on  human  factors  engineering  and  its 
relationship  to  the  thesis  work. 

Chapter  3  contains  information  on  three-dimensional  computer  graphics  displays  in 
general,  and  the  display  and  equipment  used  in  the  thesis  experiments. 

Chapter  4  discusses  current  and  projected  applications  of  three-dimensional 
perspective  displays  in  underwater  visualization. 

Chapter  5  describes  in  detail  the  conduct  of  seven  tests  including  restricting  the  field 
of  view,  grids  to  provide  depth  and  angular  cues,  color  schemes  to  provide  altitude  cues 
versus  depth  of  field,  direct  range  information,  and  slowing  of  the  graphic  display  update 
rate. 

Chapter  6  presents  the  results  of  the  seven  tests  including  operator  comments  and 
author/test  monitor  observations. 

Chapter  7  summarizes  the  results  of  the  experiments,  discusses  conclusions,  and 
offers  recommendations  for  future  work. 


16 


Chapter  2 


Human  Factors  Engineering 


2.1     Objectives  and  Doctrines 

Human  factors  engineering  is  the  study  of  people  and  how  equipment  designs  affect 
them.  The  two  major  objectives  of  human  factors  engineering  are  to  improve  efficiency 
and  effectiveness  and  to  promote  key  elements  such  as  safety,  satisfaction,  and  stress 
reduction.  Six  doctrines  that  form  a  set  of  human  factors  commandments  are  [S-l]: 

•  Design  with  the  user  in  mind. 

•  Recognize  differences  in  capabilities  and  limitations  of  people  and  their  design 

implications. 

•  Design  influences  human  behavior. 

•  Design  process  includes  empirical  data  and  evaluation. 

•  Reliance  on  scientific  methods  and  objective  data. 

•  Commitment  to  system  orientation. 


2.2     Methodology  Issues 

Experimental  research  methods  test  the  effects  of  variables  on  behavior.  Of  prime 
interest  in  this  work  is  the  behavior  "efficiency  of  operator  performance".  The  method 
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selected  needed  to  accurately  measure  this  efficiency.  Eight  important  issues  associated 
with  the  choice  of  methodology  are  [M-l]: 

•  Effectiveness. 

•  Ease  of  use. 

•  Cost. 

•  Flexibility. 

•  Range. 

•  Validity. 

•  Reliability. 

•  Objectivity. 

No  approach  was  best  for  all  the  issues  above.  Among  them  effectiveness, 
validity,  reliability,  and  objectivity  were  deemed  most  critical  for  this  thesis.  Validity,  in 
particular,  was  enhanced  by  input  from  experienced  ROV  pilots  and  operational  support 
group  personnel.  These  inputs  served  to  more  realistically  match  the  simulation  to 
operating  conditions. 


2.3     Research  Criteria 

Research  criteria  can  be  divided,  with  inevitable  overlap,  into  the  areas  of  system- 
descriptive  criteria,  task-performance  criteria,  and  human  criteria  [M-l].  The  criteria 
chosen  for  analysis  in  this  work  was  task  performance,  specifically  performance  time. 
Human  criteria  cannot  be  ignored,  however,  and  provide  insight  in  the  form  of  subjective 
comments  by  the  operators,  both  during  and  after  each  test,  and  observations  by  the  test 
monitor/author. 
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2.4     Visual-Coding 

Color  is  a  prime  visual-coding  technique.  It  has  been  frequently  used  in  3-D 
underwater  visualization  to  provide  depth/relief  cueing  in  the  form  of  a  continuous  color 
scale  versus  depth.  In  this  work  two  discrete  color  sets  were  chosen  for  testing.  Both  the 
6-  and  21-color  sets  were  derived  from  the  red,  green,  and  blue  (RGB)  color  model. 
Figure  2.1  shows  the  RGB  cube  [F-l].  The  6-color  set  was  selected  by  following  a  path 
from  corner  to  corner  in  the  following  order:  red,  yellow,  green,  cyan,  blue,  and  magenta. 
The  21-color  set  followed  the  same  path,  selecting  colors  at  each  0.25-unit  path-length 
increment. 

The  designation  of  specific  color  names  was  complicated  by  the  existence  of  a 
variety  of  descriptive  systems.  Among  the  more  notable  were  RGB,  HSB,  Munsell, 
ISCC-NBS,  YIQ,  CIE,  and  UCL  [B-l,  F-l,  K-l].  Target  color  names  were  derived 
starting  with  the  basic  six  colors  of  the  RGB  color  set  (identical  to  the  six  basic  hues  of  the 
HSB  color  set):  red,  yellow,  green,  cyan,  blue,  and  magenta  [F-l].  The  further 
subdivisions  were  named  in  accordance  with  the  spirit  of  the  Color  Naming  System 
(CNS),  which  was  developed  to  simplify  color  descriptions  [B-l].  The  21  color  names 
derived  were  used  for  the  1-,  6-,  and  21-color  sets.  Table  2.1  lists  the  colors  by  name  and 
includes  RGB  and  HSB  descriptions.  Conversions  between  most  color  sets  were  available 
and  easily  done  with  a  computer  program  [F-l]. 


2.5     Control  and  Tracking  Performance 

Controls  transmit  discrete  or  continuous  signals  to  a  system  to  produce  a  desired 
response.  Important  factors  in  control  design  include  ease  of  identification,  size,  control- 
response  ratio,  resistance,  lag,  deadspace,  backlash,  location,  and  compatibility  [S-l]. 
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Blue  -  (0,  0,  1)         Cyan  =  (0,  1,  1) 


Magenta  =  (1,  0,  1) 


Black  =  (0,  0,  0) 


White   =(1,  1,  1) 


Green  =  (0,1,  0) 


Figure  2.1 


Red  =  (1.0,  0)  Yellow  =  (1,  1,  0) 

The  RGB  (red,  green,  and  blue)  cube  [F-l] 


Table  2.1 


Color  set  RGB  to  HSB  conversion 


Color 

Red 

Green 

Blue 

Hue 

Saturation 

Brightness 

Red 

0 

0 

0 

Reddish  Orange 

0.25 

0 

15 

Orange 

0.50 

0 

30 

Yellowish  Orange 

0.75 

0 

45 

Yellow 

0 

60 

Yellowish  Yellow-Green 

0.75 

0 

75 

Yellow-Green 

0.50 

0 

90 

Greenish  Yellow-Green 

0.25 

0 

105 

Green 

0 

0 

120 

Greenish  Aquamarine 

0 

0.25 

135 

Aquamarine 

0 

0.50 

150 

Cyanish  Aquamarine 

0 

0.75 

165 

Cyan 

0 

180 

Cyanish  Turquoise 

0 

0.75 

195 

Turquoise 

0 

0.50 

210 

Blueish  Turquoise 

0 

0.25 

225 

Blue 

0 

0 

240 

Blueish  Purple 

0.25 

0 

255 

Purple 

0.50 

0 

270 

Magentaish  Purple 

0.75 

0 

285 

Magenta 

1 

0 

300 
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Deadspace  refers  to  a  null  established  about  the  control's  normal  position  in  which  control 
device  motion  results  in  no  signal  output.  Backlash  is  deadspace  at  all  control  positions. 
The  control  mechanism  may  have  no  resistance  (free-positioning  or  pure-displacement 
control),  no  displacement  (stiff  stick  or  pure-force  control),  or  lie  in  the  middle  (which  we 
term  a  mixture  control). 

The  control  order  of  a  system  refers  to  the  relationship  between  the  control 
mechanism  and  the  system  output.  Orders  of  control  include  [S-l]: 

•  position  (zero-order)  control  -  where  the  signal  controls  the  output  directly. 

•  velocity  (first-order)  control  -  where  the  signal  controls  the  rate  of  change  of 

position. 

•  acceleration  (second-order)  control  -  where  the  signal  controls  the  rate  of  change 

of  velocity. 

•  higher-order  control  -  where  the  signal  controls  the  rate  of  change  of  acceleration 

or  other  higher  order  term. 
The  system  in  these  tests  used  second-order  control  of  thrust,  equivalent  to  acceleration,  in 
each  of  the  four  degrees  of  freedom  (see  Section  5.3). 

The  problem  presented  to  the  operator  was  a  compensatory  tracking  problem  with 
time  lag  [S-l].  The  time  lag  was  composed  of  three  parts: 

•  response  lag  -  the  time  required  for  the  operator  to  choose  and  execute  a 

response. 

•  control  system  lag  -  the  time  required  for  the  system  to  respond  to  commands 

including  vehicle  motion  simulation. 

•  display  system  lag  -  the  time  required  for  graphics  calculations  and  display 

update. 
In  each  case  the  time  lag  increased  tracking  error. 
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2.6     Test  and  Evaluation 

Independent  variables  (IVs)  are  those  parameters  controlled  by  the  experimenter. 
Dependent  variables  (DVs)  are  measured  parameters  affected  by  the  IVs.  They  are  the 
same  as  the  research  criteria  discussed  in  Section  2.3. 

Evaluation  of  test  results  must  consider  test  subjects,  criteria,  and  experimental 
procedures  [S-l].  Subjects  should  be  representative  of  projected  users.  Dependent 
variables  (criteria)  must  be  related  to  system  operational  use.  Experimental  procedures  and 
controls  should  give  repeatable  results. 
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Chapter  3 


Three-Dimensional  Computer  Graphics  Displays 


3.1     True  3-D  Displays 

Technology  stands  at  the  threshold  of  true  three-dimensional  displays.  Available 
implementations  display  depth  instead  of  simply  providing  depth  cueing.  Two  such 
systems  are  SpaceGraph™  by  Bolt,  Beranek,  and  Newman  (BBN)  Laboratories 
Incorporated  of  Cambridge,  Massachusetts  and  OmniView™  by  Texas  Instruments  (TI)  of 
Dallas,  Texas. 

The  SpaceGraph"™  system  is  based  on  reflecting  a  point  plotted  (not  raster  scanned) 
image  in  a  vibrating  mirror.  The  technique  is  termed  the  varifocal  mirror.  As  the  reflecting 
surface  vibrates  it  repeatedly  changes  shape  from  concave  to  convex,  causing  a  large 
change  in  focal  length.  When  the  image  update  rate  is  synchronized  with  the  mirror 
vibration,  a  true  3-D  image  is  created  in  space.  For  many  data-set  analyses  the  superior 
pattern-recognition  capability  of  the  human  brain  is  enhanced  by  this  type  of  display.  This 
system  is  limited  in  viewing  angle,  display  update  rate,  and  to  simple  monochrome  images. 
[F-l,  T-l,  S-2] 

The  OmniView™  system  is  based  on  laser  scanning  of  a  paraboloidal,  rapidly 
rotating  surface.  During  rotation,  the  scanning  laser  beam's  reflections  form  the  3-D 
image.  Advantages  include  360°  horizontal  viewing  and  total  wrap-around  parallax.  This 
system  has  no  hidden  surface  removal  and  cannot  display  solid  surfaces.  [E-l] 
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Holography  is  another  promising  technology  for  true  3-D  viewing.  This  work  is 
based  on  the  splitting  of  a  laser  beam  into  two  parts,  illumination  of  objects  by  one  of  the 
split  beams,  and  their  subsequent  recombination  using  the  second  beam  as  a  reference. 
Limitations  typically  include  small  size  and  restricted  viewing  angle,  lack  of  parallax,  and 
high  cost.  [E-1,F-1] 

Head-mounted  displays  have  offered  tremendous  potential  since  their  birth  in  the 
late  1960s  [S-3,  S-4].  The  helmeted  user  enjoys  freedom  of  movement  in  a  virtual  3-D 
world  projected  in  real  time  for  separate  viewing  by  the  each  eye.  Technical  problems  have 
severely  restricted  the  development  of  this  technology  since  its  inception  [F-l]. 

An  inherent  limitation  of  true  three-dimensional  viewing  is  the  tremendous  quantity 
of  data  and  associated  data  processing,  particularly  critical  for  real-time  display.  Expenses 
for  computation  rise  dramatically  as  realistic,  complex  3-D  imagery  is  approached  and 
parallel-processing  supercomputers  become  a  necessity.  Significant  effort  and  funding  are 
being  funnelled  into  this  cutting-edge  technology,  but  it  is  not  likely  to  be  available  to  the 
field  scientist  before  the  late  1990s. 


3.2     Simulated  3-D  Displays 

In  the  absence  of  true  3-D  displays  several  methods  are  employed  to  simulate  three- 
dimensional  viewing  using  two-dimensional  CRTs.  These  methods  are  designed  to 
counter  the  information  loss  inherent  in  projecting  3-D  information  on  a  2-D  surface  such 
as  a  computer  screen.  Techniques  used  include  those  for  line  drawings  and  shaded  images, 
as  well  as  dynamics,  stereopsis,  and  head-motion  parallax. 

Line  drawing  techniques  include  the  use  of  various  projections,  depth  cueing  with 
intensity,  depth  clipping,  and  hidden-line  removal  [F-l].  Among  the  projections,  the 
perspective  offers  two  significant  depth  cues:  decreasing  size  of  objects  with  depth  and  the 
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convergence  of  parallel  lines  to  their  vanishing  points.  Depth  cueing  with  intensity  draws 
the  more  distant  objects  at  a  lower  intensity.  The  result  is  a  strong  depth  cue  similar  to  that 
observed  when  viewing  a  scene  in  a  light  fog.  Depth  clipping  limits  the  view  by  preventing 
the  drawing  of  lines  not  within  the  designated  limits  of  the  front  or  back  clipping  planes. 
This  effect  is  used  to  limit  the  depth  of  field.  Hidden  line  removal  calculates  the  lines  that 
should  not  be  visible  based  on  the  view  and  does  not  draw  them. 

Shaded-image  techniques  include  hidden-surface  removal,  control  of  illumination 
and  shading,  various  shading  interpolation  methods,  and  the  use  of  texture,  shadows, 
transparency,  and  reflection. 

Dynamics  refers  to  the  added  information  available  when  a  sequence  of  views  is 
shown  instead  of  a  static  view.  The  primary  depth  cue  is  provided  by  the  kinetic  depth 
effect,  the  lower  velocity  of  nearby  objects  versus  those  in  the  background. 

Stereopsis  is  a  depth  cue  taking  advantage  of  the  binocular  disparity  between  human 
eyes  [F-l].  The  ability  of  the  human  brain  to  fuse  two  slightly  differing  views  of  a  scene 
and  extract  depth  information  is  a  powerful  depth  cue. 

Head-motion  parallax  uses  the  changing  view  as  occurs  in  real  life  when  the  head  is 
moved  side  to  side. 

3.3     The  Marquest  System 

Several  important  components  of  the  test  equipment  hardware  and  software  were 
provided  by  the  Marquest  Group,  Inc.  of  Bourne,  Massachusetts.  They  include  the 
joystick  module  and  associated  software,  an  Nth  Engine/550™  Display  Controller,  and 
HOOPS™  graphics  routines  and  HYDRA™  rendering  software  for  the  Nth  Engine/550™. 
They  are  components  of  a  commercial  ROV  control  system. 

The  joystick  module  is  a  standardized  control  device  for  ROVs  constructed  by  the 
Marquest  Group,  Inc.  using  off-the-shelf  components.  Included  are  a  trackball,  bias  and 
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gain  potentiometers,  nine  push-button  switches,  and  a  3-axis  joystick.  Only  the  joystick 
was  used  in  the  testing.  Measurement  Systems,  Inc.  of  Norwalk,  Connecticut,  built  the 
industry  standard,  heavy  duty,  3-axis  (x,  y,  G)  joystick  with  mixture  control  and  deadspace 
on  each  of  the  three  axes.  A  thumb  switch  provided  the  fourth  axis  of  freedom  (z)  with 
pure  force  control.  Another  joystick  available  from  Measurement  Systems,  Inc.,  but  not 
used  here,  incorporated  the  fourth  axis  of  freedom  through  lifting/depressing  the  joystick 
vertically  with  mixture  control.  Joystick  use  is  discussed  in  Chapters  6  and  7. 

The  Nth  Engine/550™  Display  Controller  is  a  3-D  graphics  processor/controller 
built  by  Nth  Graphics  of  Austin,  Texas.  The  controller  has  1280  x  1024  resolution  and 
operates  at  20  MIPS  and  3  MFLOPS.  The  board  set  includes  an  NthTV™  converter 
module,  which  provides  a  standard  commercial  television  format  (NTSC)  tap  for  direct, 
real-time  recording  of  the  video  display  on  a  video  cassette  recorder  (VCR). 

HOOPS™  from  Ithaca  Software  of  Ithaca,  New  York,  formed  an  interactive  three- 
dimensional  graphics  library.  This  software  controlled  the  Nth  Engine/550™. 

HYDRA™  rendering  software,  another  product  of  Nth  Graphics,  was  used  to 
convert  targets  and  grids  drawn  in  AutoCAD®  Release  10  to  the  required  format  for  the 
HOOPS  graphics  routines.  A  helpful  feature  is  the  ability  to  quickly  conduct  a  visual  check 
of  targets  and  grids. 


3.4     The  Display  and  the  Monitor 

Planar  geometric  projections  fall  into  two  categories:  perspective  and  parallel. 
Characteristics  which  distinguish  a  perspective  projection  from  a  parallel  projection  are 
convergence  of  parallel  lines,  size  diminution,  and  nonuniform  foreshortening  [C-l].  The 
primary  advantage  of  a  perspective  projection  is  the  natural  appearance  of  objects  seen  by 
the  eye.  It  was  also  the  standard  video  camera  projection.  For  these  reasons  a  three-point 
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perspective  projection  was  chosen  for  the  display  [C-l].  The  graphics  were  shown  on  a 
Sony  model  GDM-1950  19-inch  color  monitor  with  1280  x  1024  pixel  resolution. 

Of  the  previously  discussed  techniques  to  provide  three  dimensionality,  depth 
cueing  in  particular,  many  were  not  available  in  the  experimental  system  for  three  reasons: 
the  tests  used  a  perspective  projection,  the  targets  were  drawn  in  wireframe  mode,  and 
limitations  of  the  Nth  Graphics  boards  and  software  running  the  display.  Depth  cueing 
was  provided  by  target  relative  size  (because  they  were  all  identical  in  dimension),  the 
convergence  of  parallel  lines  (emphasized  by  the  cubic  wireframe  targets),  and  the  kinetic 
depth  effect. 

An  NEC  VGA  color  monitor  (with  blue  background)  provided  the  following: 

•  display  update  indicated  by  the  word  "Busy.."  appearing  in  white  letters  in  a  red 

box. 

•  most  recent  target  hit  indicated  by  the  words  "OBJECT  #( target  number)  HIT" 

appearing  in  white  letters  in  a  red  box. 

•  test  completion  indicated  by  the  words  "MISSION  COMPLETE"  appearing  in 

white  letters  in  a  red  box. 

•  range  in  meters  from  the  vehicle  (modelled  as  a  point)  to  the  center  of  the  next 

target  indicated  by  the  symbol  "R  =  (range)"  appearing  in  white  letters  in  a 
red  box.  This  information  was  provided  to  the  operator  only  during  test  6. 

•  world  system  coordinates  (x,  y,  and  z)  and  heading  of  the  vehicle  in  white 

numerals.  Titles  (xpos,  ypos,  zpos,  and  hdg)  were  shown  beside  the 
coordinates/heading  in  white  letters. 

•  vehicle  system  velocities  (x,  y,  z  ,  and  6  rate)  in  white  numerals.  The  titles 

(xvel,  yvel,  zvel,  and  rvel)  shown  in  black  letters  above  the  velocities  were 
individually  accented  with  small  red  boxes  if  their  magnitudes  exceeded 
0.05  m/s  or  0.05  rad/s. 
Figure  3.1  shows  the  monitor  screen  with  range  information  during  test  6  . 
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Figure  3. 1        Monitor  screen  during  test  6 
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Chapter  4 


Application  of  the  3-D  Perspective  Display 


4.1     Ocean  Environmental  Sensing 

Early  sonar  development  was  driven  by  military  needs.  Crude  processing  and 
display  techniques  limited  these  systems  to  deteirninistic  methods  with  severely  restricted 
dynamic  range.  Inherent  in  these  limitations  were  a  disregard  for  inaccuracies  such  as 
angular  and  range  resolution,  variations  in  the  speed  of  sound  and  the  sound  ray  paths,  and 
navigation  errors  in  position,  attitude,  depth  and  motion.  The  current  state  of  technology 
overcomes  many  of  these  restrictions.  Modem  computers  and  software  graphics  packages 
allow  real-time  processing  and  display  of  more  advanced  models  of  the  ocean  environment. 
New  commercial  sonars  provide  more  accurate  information  from  a  variety  of  sampling 
techniques  including  swath  mapping,  sidescan,  and  upward-looking  sonars,  in  addition  to 
the  more  traditional  forward-looking  and  bathymetric  sounding  versions.  Innovative 
electronic  instruments  provide  high-accuracy  navigational  data.  Recent  trends  have  resulted 
in  additional  technologies  being  adapted  for  underwater  work.  Video  and  electronic  still 
cameras,  and  scanning  lasers  are  among  these  supplementary  sensors. 
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4.2     Multisensor   Modeling 

To  effectively  combine  the  various  sensor  modalities  into  a  single  model  of  the 
ocean  environment,  we  must  first  accept  the  premise  that  no  measurement  is  exact.  Any 
information  gathered  will  have  an  associated  probability  distribution,  which  is  itself  not 
known  deterministically.  The  ocean  model  can  be  built  incorporating  these  uncertainties  as 
a  stochastic  model.  This  approach  is  especially  effective  when  combining  information  from 
different  sources  with  their  differing  ranges,  resolutions,  and  data  types.  At  any  time,  a 
deterministic  model  can  be  extracted  from  the  current  probabilistic  one.  Using  this  method, 
a  more  accurate  deterministic  model  is  obtained  which  accounts  for  the  many  and  varied 
errors  inherent  in  measurements. 

Initially,  the  ocean  volume  of  interest  is  divided  into  a  3-D  grid  of  cubic  volume 
elements,  or  voxels.  The  size  of  the  voxels  is  dependent  upon  the  desired  model  resolution 
as  well  as  system  hardware  and  software  limitations  for  real-time  processing.  Each  voxel 
is  assigned  a  "feature"  vector  incorporating  information  such  as  the  time,  the  position  of  the 
voxel  in  the  coordinate  space,  the  measured  characteristics  of  interest  (features),  and  a 
probability  or  confidence  value  for  each  feature.  As  new  information  is  collected,  older 
data  is  not  discarded.  Feature  vectors  are  incremented  using  a  data  merging  technique 
called  stochastic  backprojection  [S-5,  S-6].  This  data  merging  technique  is  an  adaptation  of 
the  backprojection  and  summation  method  used  to  reconstruct  images.  The  method  is 
enhanced  to  include  probabilistic  uncertainty  and  allow  stepwise  solutions  for  real-time 
processing.  Use  of  these  techniques  allows  combination  of  the  redundant  feature  data  from 
all  available  sources,  improving  the  model  resolution  and  certainty. 


30 


4.3     The  Deep  Submergence  Laboratory  System 

The  Deep  Submergence  Laboratory  of  the  Woods  Hole  Oceanographic  Institute 
employs  a  variety  of  sensors  and  platforms.  A  typical  cruise  involves  flying  equipment  to  a 
proximate  port  and  loading  out  a  chartered  mother  vessel.  At  the  research  site,  a  large  area 
survey  is  conducted  with  a  towed  120-kHz  split-beam  sonar  system.  This  survey  provides 
medium  resolution  imaging  with  a  swath  width  of  200-300  meters  at  a  tow  speed  of  6-10 
knots,  and  allows  items  of  interest  to  be  identified  for  subsequent  revisiting  with  higher 
resolution  equipment.  Items  of  interest  are  reinspected  using  the  tethered  Jason  ROV 
system.  Jason  is  typically  equipped  with  a  SHARPS  short-baseline  navigation  system  and 
a  sensor  package  including  a  200-kHz  sidescan  sonar,  a  300-kHz  forward-scanning  sonar, 
a  video  camera,  an  electronic  still  camera,  and  a  laser  rangefinder.  Figure  4.1  shows  the 
range  and  resolution  overlaps  between  sensors  in  the  current  suite. 

The  Jason  pilot  has  several  sources  of  information  available.  If  within  visual  range, 
the  video  camera  and  vehicle  Ugh  ting  system  provide  a  natural,  perspective  view  with 
extremely  high  bandwidth.  This  visual  range  is  often  restricted  to  a  meter  or  less  due  to 
suspended  sediment  in  the  water,  and  is  usually  less  than  about  10  meters.  Beyond  visual 
range  the  forward- scanning  sonar,  the  200-kHz  sidescan  sonar,  the  laser  rangefinder,  and 
the  SHARPS  navigation  system  provide  data  to  several  displays  with  differing  formats. 
The  pilot  navigates  using  a  specially  designed  2-D  auxiliary  display  with  limited  bandwidth 
until  within  visual  range  [S-7]. 

This  current  method  taxes  the  pilot,  whose  task  is  complicated  by  [S-7]: 

•  low  visibility,  reducing  video  camera  range  and  safe  working  speeds. 

•  the  narrow  field  of  view  of  the  video  camera  (approximately  40°). 

•  the  monocular  vision  of  the  video  camera. 

•  optical  distortion  of  the  video  camera. 

•  the  difficulty  of  interpreting  sonar  presentation  screens. 
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Figure  4. 1        DSL  comparison  of  underwater  remote-sensing  systems 
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•  the  transient  nature  of  the  displayed  information,  requiring  the  pilot  to  mentally 

integrate  the  overall  picture. 

•  the  inability  to  access  the  high-level,  real-time  environmental  model. 

To  enhance  ROV  piloting  efficiency,  a  real-time,  3-D  display  is  envisioned.  The 
display  will  access  the  high-level  environmental  model  and  present  the  user  with  a  view 
closely  resembling  that  of  the  video  camera.  Because  low  or  medium  resolution  data  are 
available  prior  to  the  ROV  entering  the  water,  a  rough  perspective- view  display  can  be 
constructed  at  any  time  for  use  by  the  pilot  on  the  screen  previously  reserved  for  video 
camera  information.  Such  a  perspective-view  display  is  expected  to  be  dominated  in  the 
foreground  by  high-resolution  video  data,  gradually  transitioning  to  dominance  by  other 
sensors  as  the  limits  of  video  range  are  encountered. 


4.4     Targeted  Human  Factors 

Among  the  myriad  of  potential  display  enhancements  to  improve  the  proposed  3-D 
perspective  display  are: 

•  altering  the  displayed  field  of  view. 

•  providing  a  screen  grid. 

•  displaying  current  range  to  the  target  of  interest. 

•  use  of  a  vertical  color  scheme. 

•  controlling  the  display  update  rate. 

Narrowing  the  displayed  field  of  view  reduces  the  necessary  graphics  calculations 
as  the  square  of  the  reduction  factor.  This  gain  must  be  balanced  against  the  loss  of 
information  to  the  user. 
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Providing  a  2-D  or  3-D  screen  grid  can  improve  the  user's  angular  perception, 
provide  a  cross-hair  effect  to  ease  direction  finding  and,  with  the  3-D  version,  provide 
important  depth  cues. 

Displaying  current  range  to  the  target  of  interest  can  reduce  the  time  to  locate  the 
target.  This  information  is  available  in  the  environmental  model. 

Vertical  color  schemes  are  frequently  used  to  provide  redundant  relief  cueing  for 
underwater  and  land  feature  presentations  such  as  charts  and  maps.  They  can  be  effective 
in  giving  altitude  cues  for  objects  in  the  background  of  perspective  displays. 

Display  update  rate  is  limited  by  the  data-collection  cycle  time  of  the  sensors,  data 
processing  time,  and  graphics  calculation/display  time.  Compromises  can  be  made  to 
shorten  the  update  rate  at  the  cost  of  information  quality  and  quantity. 

This  thesis  attempts  to  quantify  the  effects  of  the  task-related  independent  variables 
listed  above.  The  dependent  variables  for  the  experiments  are  performance  time,  subjective 
comments  by  the  operators,  and  observations  by  the  test  monitor/author. 
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Chapter  5 


Testing  Display  Effectiveness 


5.1     Equipment 

Three  system  hardware  components  were  used  directly  by  the  operator:  a  computer 
monitor,  a  computer  display,  and  a  joystick.  Other  equipment  transparent  to  the  operator's 
task  included  a  386  computer  with  keyboard  and  mouse,  which  ran  the  simulation  and 
recorded  the  data,  and  various  power  supply  and  communications  cabling. 


5.2       Data  Point  Courses 

Nineteen  data  courses  (A-R,  X)  were  used  in  testing.  Courses  consisted  of  25 
sequentially  numbered  data  points  (Pj)  in  world  coordinate  system  three  space  (x,  y,  z) 

with  two  constraints: 

(1)  Consecutive  points  were  separated  by  distance  s  =  v  [Ax]2+[Ay]2+[Az]2,  where 

s  was  a  constant  equal  to  5  meters. 

(2)  A  square  window  of  given  total  angular  width  and  height  (3,  with  apex  at  point 

Pj  and  opening  in  the  direction  of  a  line  from  Pj.j  through  Pj,  contained 
point  Pj+1. 

Courses  A,  B,  C  and  J,  K,  L  used  a  15°- square  window,  courses  D,  E,  F  and  M,  N,  O 
used  a  30°  window,  and  courses  G,  H,  I  and  P,  Q,  R  used  a  60°  window.  Course  X,  the 
training/famiUarization  course,  used  a  15°  window.  Additionally,  nine  of  the  courses  were 
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mirror  images  of  each  other  in  the  world  coordinate  system  y/z  plane.  Mirror  pairs  were 
A/J,  B/K,  C/L,  D/M,  E/N,  F/O,  G/P,  H/Q  and  VR.  Figure  5.1  and  table  5.1  summarize 
the  above  information. 

Visualization  of  each  data  point  was  provided  in  the  form  of  a  cubic  wireframe 
target  one  meter  on  a  side  with  the  invisible  data  point  at  its  center.  Target  faces  were 
parallel  to  the  planes  formed  by  the  world  coordinate  system  axes.  Target  color  was  blue 
except  for  tests  5  and  6  as  discussed  in  section  5.7.  Each  target  was  numbered  with  white 
numerals  (1-25)  corresponding  to  the  sequential  number  of  the  data  point  at  its  center.  A 
target  "hit"  corresponded  to  closing  the  range  to  the  target  center/data  point  to  0.5  meters  or 
less  from  any  direction.  Appendix  C  discusses  the  details  of  target  design.  Appendix  D 
contains  course  layout  data  and  photographs  of  the  courses  as  viewed  by  the  operator. 

Random  generation  of  the  data  points  for  each  course  began  with  establishment  of 
two  coordinate  systems,  the  world  and  the  vehicle.  Each  system  consisted  of  three 
mutually  perpendicular  axes  x,  y,  and  z,  forming  a  right-hand  coordinate  system.  The 
world  system  was  fixed  in  space.  In  it,  three  angles  were  defined:  CO,  cp,  and  k  for  positive 
rotation  about  the  x,  y  and  z  axes  respectively.  Positive  rotation  was  defined  such  that 
positive  co  (cp,  k)  rotated  the  positive  y  (z,  x)  axis  into  the  positive  z  (x,  y)  axis  through  90°. 
In  the  vehicle  system,  the  operator  was  assumed  to  be  looking  down  the  positive  y  axis  of 
the  system  at  all  times  with  the  positive  x  axis  to  the  right.  Positive  rotation  was  defined  so 
positive  0  rotated  the  positive  y  axis  into  the  positive  x  axis  through  90°.  In  this  way,  0 
had  the  same  sense  as  true  heading.  Figures  5.2  and  5.3  depict  these  systems.  For  a 
specified  window  width  (3,  sets  of  two  angles  co  and  k  were  randomly  selected  from  the 
interval  [-[3/2,  (3/2].  This  process  insured  that  the  data  points  met  constraint  (2)  above. 

Based  on  the  above  coordinate  system  definitions,  rotations  of  the  vehicle  system 
were  broken  down  into  individual  rotations  of  co,  (p,  and  k.  Orthogonal  orientation 
matrices  were  then  defined  for  the  decomposition  as  follows  [M-2]: 
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-sin  co 
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-sin  cp 
0 
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COS    K 

sin  K 

0 

-sin  k 
0 

COS    K 

0 

0 
1 

The  most  general  orientation  matrix  M  for  any  combination  of  these  three  rotations 
was  then  [M-2]: 


M  =  MC0M{pMK 


cos  cp  cos  k      cos  co  sin  k  +  sin  co  sin  cp  cos  k 

-cos  cp  sin  k       cos  co  cos  k  -  sin  co  sin  cp  sin  k 

sin  cp  -sin  CO   cos  cp 

sin  co  sin  k  -  cos  co  sin  cp  cos  k 
sin  co  cos  K  +  cos  co  sin  cp  sin  k 
cos  co  cos  cp 


In  our  application  pitch  (co)  and  yaw  (k)  were  allowed  to  vary  while  roll  (cp)  was 
not.  The  general  case  was  then  specialized  by  setting  cp  =  0  to  obtain: 


COS    K 

cos  co  sin  k 

sin  co  sin  k 

L  =  Mcp=0  = 

-sin  k 

COS   CO    COS    K 

sin  co  cos  k 

0 

-sin  co 

COS    CO 

The  matrix  Li  defined  the  look  direction  from  the  point  P;  to  the  next  point  Pj+j  in 
the  vehicle  coordinate  system.  The  matrix  (Lj)    defined  the  transformation  from  the  vehicle 
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coordinate  system  to  that  of  the  world.  The  origin  point  was  taken  to  be  Pq  =  [  0,0,0  ]™. 
The  first  data  point  was  taken  to  be  Pi  =  s[  0,1,0  ]*,  where  s  was  the  constant  described  in 
constraint  (1).  The  value  (0,  0)  was  assigned  to  the  set  (co,  k)  at  x  =  0  yielding  Lq  =  I3X3> 
the  three-dimensional  identity  matrix,  and  successive  Ljs  were  generated  using  the  above 

formula  for  matrix  L  and  the  j-th  set  of  values  (co,  k).  As  a  result  of  these  specifications 
(Lq)    =  l3x3  and  successive  (Lj)  's  were  calculated  from  the  equation  (Lj+j)    =  (L;)  Lj. 
Successive  data  points  were  then  generated  using  Pj+i  =  Pj  +  s(Lj)  Lj  [0,1,0  ]T  as  the 

recursive  algorithm. 


5.3     Modeling  the  Jason  Vehicle 

The  vehicle  model  was  a  dynamic  simulation  of  the  DSL  ROV  Jason.  The  model 
exhibited  four  degrees  of  freedom:  x  (left/right),  y  (forward/reverse),  z  (up/down),  and  9 
(CW/CCW).  Hydrodynamic  drag  was  assumed  to  have  quadratic  and  linear  (damping) 
components.  Balancing  forces  yielded  the  equation  (shown  for  the  x  component) : 


(M  +  Ma)x  + 


(M  +  Ma) 


x  +  Cq  x  I  x  I    =   T 


xmax 

where, 

M         was  the  vehicle's  mass,  Ma  was  the  vehicle's  added  mass,  and  (M  +  Ma) 

was  termed  the  vehicle's  extended  mass. 
xmax    was  a  constant  (2.0  s)  defining  the  magnitude  of  the  linear  drag. 
Crj       was  the  drag  coefficient. 

T  was  the  maximum  thrust. 

Table  5.2  summarizes  the  model  parameters  use  in  the  experiment.  Nonlinear 
thruster  dynamics  were  not  included  in  the  vehicle  simulation  model. 
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The  square  window  actually  defined  a  portion  of  the 
surface  of  a  sphere  because  point  P\+\  was  required 

to  be  exactly  distance  s  from  point  Pj. 

/ 

>1 


<-. 


pi-i 


=pj+i 

window 


Figure  5. 1        Visualization  of  data  point  course  constraints 


Table  5. 1  Data  point  course  relationships 


15°  Solid  Angle 

30°  Solid  Angle 

60°  Solid  Angle 

Image 
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B 
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E 

F 

G 

H 

I 

Mirror 

J 

K 

L 

M 

N 

O 

P 

Q 

R 

Training 

X 

Table  5.2         Dynamic  vehicle  simulation  parameters 


Extended  Mass 
(kg) 

Drag  Coefficient 
(kg/m  or  kg-m) 

Max  Thrust 
(N) 

Max  Velocity 
(m/s  or  rad/s) 

X 

1800 

400 

250 

0.25 

y 

1350 

350 

250 

0.50 

z 

1800 

400 

250 

0.25 

e 

450 

250 

41.5 

0.157 
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Figure  5.2        The  world  coordinate  system 
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Figure  5.3        The  vehicle  coordinate  system 
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5.4     Operators  and  Training 

The  same  four  operators  were  used  in  each  of  the  seven  tests.  The  author  was  not 
an  operator.  Operator  profiles  were: 

Operator  I  was  a  22  year  old,  male  Naval  Officer  and  graduate  student  with 
uncorrected  full  visual  acuity  and  normal  color  vision.  This  right-handed  subject 
had  frequently  played  video  games  including  flight  simulators.  He  was  familiar 
with  3-D  perspective  computer  displays  prior  to  the  experiment. 

Operator  II  was  a  24  year  old,  male  graduate  student  with  uncorrected  full  visual 
acuity  and  normal  color  vision.  This  left-handed  subject  had  little  exposure  to  video 
games  such  as  flight  simulators.  He  was  familiar  with  3-D  perspective  computer 
displays  prior  to  the  experiment  due  to  practical  experience  with  computer  systems 
as  an  undergraduate  Computer  Science  major. 

Operator  III  was  a  34  year  old,  male  Naval  Officer  and  graduate  student  with 
uncorrected  full  visual  acuity  and  normal  color  vision.  This  right-handed  subject 
had  some  exposure  to  video  games  such  as  flight  simulators.  He  was  familiar  with 
3-D  perspective  computer  displays  prior  to  the  experiment. 

Operator  IV  was  a  32  year  old  male  with  a  Master's  Degree,  uncorrected  full  visual 
acuity,  and  normal  color  vision.  This  right-handed  subject  had  little  exposure  to 
video  games  such  as  flight  simulators.  He  was  not  familiar  with  3-D  perspective 
computer  displays  prior  to  the  experiment. 

Prior  to  each  testing  session,  operators  were  briefed  in  the  following  areas: 
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general,  simulation,  and  hardware.  The  general  topics  included  purpose  of  thesis  and 
experimental  approach,  time  measurement,  and  operator  errors.  Simulation  topics  included 
target  and  course  design  as  well  as  vehicle  modeling.  Hardware  subjects  discussed  impor- 
tant features  of  the  computer  monitor,  the  computer  display,  and  the  joystick.  Additionally, 
a  maximum  of  15  minutes  was  allotted  for  familiarity  training  with  a  60°,  30°,  or  15°  field 
of  view,  as  appropriate  to  the  test,  using  training  course  X. 

The  tests  were  conducted  in  a  total  of  four  separate  sittings  to  reduce  operator 
fatigue.  Test  1  (see  number  designations  below)  was  conducted  alone.  The  remaining 
tests  were  conducted  in  pairs  in  the  following  order:  2/3,  4/5,  and  6/7. 


5.5     Test  1  -  Field  of  View 

Testing  for  field  of  view  measured  changes  in  the  operator's  spatial  performance 
versus  the  displayed  field  of  view. 

Test  1  description:  All  targets  were  colored  blue.  The  operators  drove  the  simulated 
vehicle  point  to  point  over  a  course  of  25  data  points  spaced  in  x,  y,  and  z.  Operators 
completed  nine  courses,  each  varying  in  x/y/z  with  a  maximum  15°,  30°,  or  60°  spread  in 
data  from  point  to  point.  Measured  fields  of  view  were  15°,  30°,  and  60°.  The  display 
update  period  was  fixed  at  two  seconds.  Nine  combinations  of  FOV  and  courses  were 
used  with  each  of  the  four  operators. 


5.6     Tests  2  and  3  -  Gridding 

Testing  for  gridding  measured  the  improvement  in  operator  performance  when 
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presented  with  a  visual  reference  grid  in  the  display. 

Test  2  description:  All  targets  were  colored  blue.  The  operators  drove  the  simulated 
vehicle  point  to  point  over  a  course  of  25  data  points  spaced  in  x,  y,  and  z.  Operators 
completed  three  courses,  each  varying  in  x/y/z  with  a  maximum  15°,  30°,  or  60°  spread  in 
data  from  point  to  point.  The  field  of  view  was  fixed  at  60°  and  the  display  update  period  at 
two  seconds.  A  two-dimensional  (2-D)  grid  was  superimposed  on  the  visual  display. 

Test  3  description:  All  targets  were  colored  blue.  The  operators  drove  the  simulated 
vehicle  point  to  point  over  a  course  of  25  data  points  spaced  in  x,  y,  and  z.  Operators 
completed  three  courses,  each  varying  in  x/y/z  with  a  maximum  15°,  30°,  or  60°  spread  in 
data  from  point  to  point.  The  field  of  view  was  fixed  at  60°  and  the  display  update  period  at 
two  seconds.  A  three-dimensional  (3-D)  grid  was  superimposed  on  the  visual  display. 


5.7     Tests  4  and  5  -  Vertical  Color  Schemes 

Testing  for  color  scheme  measured  the  improvement  in  operator  performance  using 
a  fixed,  vertical-relief  color  scheme  for  target  shading  versus  single-color  targets.  The  6- 
color  set  provided  12  meter  vertical  color  swaths  while  the  21 -color  set  provided  three 
meter  vertical  color  swaths  over  the  range  of  interest.  Table  5.3  lists  the  color  sets. 

Test  4  description:  Targets  were  shaded  in  color  according  to  height  z  above  ocean  floor. 
The  color  scheme  was  divided  into  six  discrete  shades.  The  operators  drove  the  simulated 
vehicle  point  to  point  over  a  course  of  25  data  points  spaced  in  x,  y,  and  z.  Operators 
completed  three  courses,  each  varying  in  x/y/z  with  a  maximum  15°,  30°,  or  60°  spread  in 
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Table  5.3         The  color  sets  (1,  6,  21)  and  associated  altitude  bands 


Set 

Target 

Color 

Red 

Green 

Blue 

Z  value  (meters) 

1 

0 

Blue 

0 

0 

1 

All 

6 

1 

Red 

1 

0 

0 

<88 

6 

2 

Yellow 

1 

1 

0 

76-88 

6 

3 

Green 

0 

1 

0 

64-76 

6 

4 

Cyan 

0 

1 

1 

52-64 

6 

5 

Blue 

0 

0 

1 

40-52 

6 

6 

Magenta 

0 

1 

<40 

21 

7 

Red 

0 

0 

>93 

21 

8 

Reddish  Orange 

0.25 

0 

90-93 

21 

9 

Orange 

0.50 

0 

87-90 

21 

10 

Yellowish  Orange 

0.75 

0 

84-87 

21 

11 

Yellow 

0 

81-84 

21 

12 

Yellowish  Yellow-Green 

0.75 

0 

78-81 

21 

13 

Yellow-Green 

0.50 

0 

75-78 

21 

14 

Greenish  Yellow-Green 

0.25 

0 

72-75 

21 

15 

Green 

0 

0 

69-72 

21 

16 

Greenish  Aquamarine 

0 

0.25 

66-69 

21 

17 

Aquamarine 

0 

0.50 

63-66 

21 

18 

Cyanish  Aquamarine 

0 

0.75 

60-63 

21 

19 

Cyan 

0 

57-60 

21 

20 

Cyanish  Turquoise 

0 

0.75 

54-57 

21 

21 

Turquoise 

0 

0.50 

51-54 

21 

22 

Blueish  Turquoise 

0 

0.25 

48-51 

21 

23 

Blue 

0 

0 

45-48 

21 

24 

Blueish  Purple 

0.25 

0 

42-45 

21 

25 

Purple 

0.50 

0 

39-42 

21 

26 

Magentaish  Purple 

0.75 

0 

36-39 

21 

27 

Magenta 

1 

0 

<36 
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data  from  point  to  point.  The  field  of  view  was  fixed  at  60°  and  the  display  update  period  at 
two  seconds. 

Test  5  description:  Targets  were  shaded  in  color  according  to  height  z  above  ocean  floor. 
The  color  scheme  was  divided  into  21  discrete  shades.  The  operators  drove  the  simulated 
vehicle  point  to  point  over  a  course  of  25  data  points  spaced  in  x,  y,  and  z.  Operators 
completed  three  courses,  each  varying  in  x/y/z  with  a  maximum  15°,  30°,  or  60°  spread  in 
data  from  point  to  point.  The  field  of  view  was  fixed  at  60°  and  the  display  update  period  at 
two  seconds. 


5.8     Test  6  -  Heads  Up  Range 

Testing  for  range  measured  the  improvement  in  operator  performance  when 
presented  with  continuously  revised  range  to  target  center  information. 

Test  6  description:  All  targets  were  colored  blue.  The  operators  drove  the  simulated 
vehicle  point  to  point  over  a  course  of  25  data  points  spaced  in  x,  y,  and  z.  Operators 
completed  three  courses,  each  varying  in  x/y/z  with  a  maximum  15°,  30°,  or  60°  spread  in 
data  from  point  to  point.  The  field  of  view  was  fixed  at  60°  and  the  display  update  period  at 
two  seconds.  Continuously  revised  range  to  target  center  was  provided  to  the  operator. 


5.9     Test  7  -  Display  Update  Rate 

Testing  for  display  update  rate  measured  the  degradation  in  operator  performance 
associated  with  reduced  visual  information  feedback. 
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Test  7  description:  All  targets  were  colored  blue.  The  operators  drove  the  simulated 
vehicle  point  to  point  over  a  course  of  25  data  points  spaced  in  x,  y,  and  z.  Operators 
completed  three  courses,  each  varying  in  x/y/z  with  a  maximum  15°,  30°,  or  60°  spread  in 
data  from  point  to  point.  The  field  of  view  was  fixed  at  60°.    The  display  update  period 
was  slowed  to  four  seconds. 
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Chapter  6 


Testing  Results 


6.1     General 

The  first  step  in  the  analysis  was  to  convert  the  logged  target  hit  times  into  time 
differences  between  successive  targets.  Appendix  E  lists  the  resultant  time  differences. 
The  second  step  was  normalize  the  raw  data  from  Appendix  E  by  removing  the  effects 
properly  attributed  to  variations  in  the  level  of  course  difficulty  related  to  the  Jason  vehicle 
simulation  model.  This  procedure  divided  each  time  difference  (in  seconds)  by  the  best 
time  (in  seconds)  that  could  be  made  between  the  associated  targets  based  on  the  vehicle 
simulation.  Appendix  D  discusses  the  method  of  arriving  at  this  best  time  between  targets 
and  lists  those  times  in  table  form.  Appendix  F  contains  the  normalized,  dimensionless 
values.  The  third  step  was  to  estimate  the  operators'  learning  curves  to  standardize  results 
between  tests.  These  results  were  then  analyzed  directly  to  assess  the  effectiveness  of  the 
tested  display  enhancements. 


6.2     Operator  Learning  Curves 

Operator  learning  reflected  a  number  of  factors  including  increasing  familiarity  with 
test  equipment,  improving  knowledge  of  test  procedures,  and  gradual  easing  of  pretest 
anxiety,  as  well  as  modified  strategies  for  driving  the  simulated  vehicle.  Estimation  of  the 
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operators'  learning  curves  involved  four  assumptions. 

•  The  learning  curves  had  positive  slope  at  all  times.  This  meant  that  the  operators 

were  constantly  improving  with  no  loss  of  knowledge  or  acquired  skills 
between  testing  sessions. 

•  The  learning  curves  were  smooth  functions.  This  prohibited  the  consideration  of 

learning  plateaus  during  which  operators  would  show  no  improvement. 

•  The  learning  curves  were  exponential  functions. 

•  For  a  specified  field  of  view,  operator  learning  was  reflected  primarily  in  the 

standard  deviation  of  the  normalized  course  times  (a  smoothness  measure). 

The  effectiveness  of  display  enhancements  was  reflected  primarily  in  the 

total  course  times  (a  measure  of  speed). 
As  an  additional  consideration,  test  1  was  designed  to  force  the  operators  to  use  a  variety  of 
strategies,  particularly  with  the  reduced  fields  of  view,  in  an  attempt  to  ensure  maximum 
learning  in  minimum  time. 

Each  operator  drove  21  courses  with  a  60°  field  of  view,  three  for  each  of  the  seven 
tests.  The  procedure  below  was  applied  to  the  normalized  data  for  each  operator  to  estimate 
his  60°  FOV  learning  curve. 

•  The  standard  deviations  were  entered  into  a  3  x  7  matrix.  Each  row  represented 

a  specific  window  width  p  as  discussed  in  section  5.2  (row  1  =  15°,  row  2 
=  30°,  and  row  3  =  60°).  Each  column  corresponded  to  a  given  test  (column 
1  =  test  1,  column  2  =  test  2,  etc.). 

•  The  standard  deviations  for  the  three  window  widths  for  each  test  were  summed 

and  entered  into  a  row  vector  of  seven  elements.  Each  column  continued  to 
represent  a  given  test  as  above.  This  step  yielded  the  summed  standard 
deviations  (SSD)  shown  in  table  6.1. 

•  The  row  vector  elements  were  plotted  with  summed  standard  deviations  on  the 

vertical  axis  versus  test  number  on  the  horizontal  axis. 
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•  An  exponential  curve  using  the  method  of  least  squares  was  fitted  to  the  plot  by 

linearizing  the  exponential  form  through  taking  logarithms  [G-l].  This 
defined  the  operator's  learning  curve.  Appendix  G  shows  the  graphed 
operators'  learning  curves. 
Once  the  operators'  learning  curves  were  estimated,  correction  factors  to  allow 
results  between  tests  to  be  compared  were  derived  using  the  following  procedure. 

•  Estimated  standard  deviation  values  were  extracted  from  the  fitted  exponential 

curve  using  regression  techniques.  This  step  produced  the  extracted  values 
(EV)  listed  in  table  6.1. 

•  Standardization  factors  (SF)  were  formed  by  dividing  all  EVs  by  the  first  (and 

largest)  value.  The  EVs  are  listed  in  table  6.1. 

•  The  84  mean  values  of  the  normalized  60°  FOV  course  times  listed  in  Appendix  F 

were  multiplied  by  the  associated  SF.  This  step  resulted  in  the  learning- 
corrected,  normalized  mean  times  (LCNMT)  listed  in  table  6.2. 

•  For  each  test,  the  LCNMTs  for  all  operators  were  summed.  The  mean  and 

standard  deviation  of  each  LCNMT  sum  were  found,  completing  the  data 

manipulation. 
Figure  6.1  shows  the  plotted  means  and  standard  deviations  of  the  normalized  mean  times 
without  correction  for  individual  operator  learning  curves.  Figure  6.2  shows  the  means 
and  standard  deviations  of  the  LCNMT  sum  plotted  for  analysis. 

The  learning  curve  for  the  different  fields  of  view  could  not  be  estimated  using  the 
same  techniques.  It  was  assumed  that  the  individual  operator's  60°  FOV  learning  curve 
was  a  good  approximation  and  the  first  three  SFs  from  table  6. 1  for  each  operator  were 
applied  to  the  data. 
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Table  6. 1  Summed  standard  deviations  (SSD),  extracted  values  (EV),  and 

standardization  factors  (SF) 


Operator  I 

O 

serator  II 

Operator  III 

Operator  IV 

Test 

SSD 

EV 

SF 

SSD 

EV 

SF 

SSD 

EV 

SF 

SSD 

EV 

SF 

1 

4.16 

3.56 

1.00 

7.65 

5.86 

1.00 

7.35 

5.32 

1.00 

3.85 

3.51 

1.00 

2 

3.01 

3.39 

1.05 

5.55 

5.12 

1.15 

3.72 

5.01 

1.06 

3.09 

3.48 

1.01 

3 

3.36 

3.22 

1.11 

4.41 

4.47 

1.31 

5.70 

4.71 

1.13 

3.48 

3.46 

1.01 

4 

2.73 

3.06 

1.17 

2.36 

3.90 

1.50 

3.47 

4.43 

1.20 

3.20 

3.43 

1.02 

5 

2.75 

2.91 

1.23 

2.66 

3.40 

1.72 

3.45 

4.17 

1.28 

3.59 

3.41 

1.03 

6 

2.53 

2.76 

1.29 

3.31 

2.97 

1.97 

3.82 

3.92 

1.36 

3.67 

3.38 

1.04 

7 

3.10 

2.62 

1.36 

3.56 

2.59 

2.26 

4.82 

3.69 

1.44 

3.20 

3.36 

1.04 

Table  6.2         Learning-corrected,  normalized  mean  times  (60°  FOV) 


Operator  I 

O 

Derator  II 

Operator  ] 

11 

Operator 

[V 

Test 

15°  p 

30°  p 

60°  p 

15°  p 

30°  p 

60°  P 

15°  p 

30°  p 

60°  P 

15°  p 

30°  P 

60°  P 

1 

1.84 

2.26 

3.62 

3.00 

2.90 

5.55 

2.52 

3.32 

3.85 

3.70 

3.15 

3.48 

2 

2.18 

2.38 

3.65 

2.10 

3.47 

4.72 

2.91 

2.32 

3.77 

2.18 

1.93 

3.03 

3 

1.90 

1.85 

3.26 

3.49 

4.12 

4.65 

3.09 

3.26 

2.81 

1.61 

1.94 

2.95 

4 

1.71 

1.94 

3.09 

2.31 

2.68 

4.57 

2.74 

2.20 

4.20 

1.98 

1.81 

3.11 

5 

2.65 

2.02 

3.43 

3.23 

3.68 

4.78 

2.98 

2.69 

3.11 

1.61 

1.56 

2.93 

6 

1.81 

2.40 

3.61 

3.94 

3.67 

7.77 

2.12 

4.02 

4.68 

1.66 

2.40 

3.14 

7 

2.71 

3.60 

3.97 

6.33 

5.87 

7.81 

3.58 

4.66 

5.63 

2.06 

2.20 

3.40 

50 
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Figure  6.1        Normalized  mean  times  (60°  FOV) 
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Figure  6.2        Learning-corrected,  normalized  mean  times  (60°  FOV) 
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6.3     Field  of  View 

The  three  fields  of  view  used  in  testing  were  related  by  inverse  powers  of  two  (60°, 
30°,  and  15°).  Plotted  means  and  standard  deviations  of  the  learning-curve-corrected  data 
versus  FOV  scaled  by  inverse  powers  of  two  were  virtually  linear  and  clearly  demonstrated 
the  parabolic  relationship  between  the  mean  run  time  and  the  FOV.  Fitting  a  parabola  to  the 
data  showed  that: 

normalized  mean  run  time  =  10.399  -  0.200  *  FOV  +  1.350  *  10'3  *  FOV2. 


6.4     Gridding 

The  2-D  and  3-D  grids  improved  operator  performance  to  the  same  degree  (34.64 
and  34.92  respectively).  The  2-D  grid  was  simple  and  straightforward  to  employ.  The  3- 
D  grid  conveyed  more  information,  but  cluttered  the  display  and  was  harder  to  interpret. 
The  primary  benefit  of  the  2-D  display  was  the  "cross-hairs"  effect  of  the  grid,  which  aided 
steering  effectiveness.  This  effect  was  less  pronounced  with  the  3-D  grid.  Flidden-line 
removal,  particularly  for  a  3-D  grid,  could  provide  strong  visual  range  cues  in  the  vicinity 
of  a  target. 


6.5     Vertical  Color  Schemes 

The  6-color  scheme  and  the  21 -color  scheme  both  provided  improved  operator 
performance.  The  6-color  scheme  proved  to  be  slightly  better  (32.34  versus  34.68).  This 
effect  was  attributed  to  its  simplicity  and  the  high-contrast,  easily  distinguished  colors.  The 
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Table  6.3  Learning-corrected,  normalized  mean  times  (varying  FOV) 


Operator  I 

Operator  II 

Operator  III 

Operator  IV 

FOV 

15°  p 

30°  p 

60°  (3 

15°  p 

30°  P 

60°  p 

15°  p 

30°  p 

60°  p 

15°  p 

30°  p 

60°  P 

60 

1.84 

2.26 

3.62 

3.00 

2.90 

5.55 

2.52 

3.32 

3.85 

3.70 

3.15 

3.48 

30 

5.89 

5.07 

4.99 

4.63 

4.73 

6.05 

6.49 

5.71 

7.13 

3.80 

4.11 

8.83 

15 

5.97 

4.96 

7.17 

7.30 

10.7 

11.6 

2.28 

5.63 

11.1 

6.75 

8.45 

10.5 

12 


mean  ±  a         6 


i 

{ 

0 

FOV->  60  30  15 

Figure  6.3        Learning-corrected,  normalized  mean  times  (varying  FOV) 
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primary  benefit  of  the  color  schemes  was  improved  spatial  perception  of  the  display  due  to 
the  combined  effects  of  depth  and  altitude  cues.  These  two  cues  were  strongly  interrelated. 


6.6     Heads-Up  Range 

A  surprising  result  was  the  degradation  of  operator  performance  when  supplied 
with  range  to  the  target  center.  Although  the  range  information  proved  valuable  if  a  target 
was  not  visible,  was  close,  or  had  been  missed,  this  was  more  than  offset  by  negative 
factors.  Primary  problems  included  ignoring  the  visual  information  of  the  display  and  the 
"perigee  effect,"  where  a  control  action  that  caused  the  range  to  initially  decrease  would  not 
necessarily  result  in  a  hit.  As  the  vehicle's  trajectory  passed  perigee  with  the  target's 
center,  the  range  increased,  frequently  confusing  the  operator.  No  individual  operator's 
performance  improved  with  range  information. 


6.7     Display  Update  Rate 

As  anticipated,  slowing  the  graphics  display  update  rate  by  a  factor  of  two  hurt 
operator  performance.  The  magnitude  of  this  effect,  slowing  the  normalized  run  time  by  a 
factor  of  1.3,  was  smaller  than  anticipated.  This  had  significant  implications  for  field 
applications,  where  a  longer  update  period  would  allow  the  use  of  less  powerful  computer 
equipment  or  more  robust  programming.  The  loss  of  the  operator's  ability  to  mentally 
integrate  the  visual  information  from  the  display  and  use  kinetic  depth-effect  cues  was 
mitigated  by  the  more  conservative  driving  approach  adopted  by  the  operators.  All  the 
operators  demonstrated  this  slowing  effect  with  update  rate. 
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6.8     Operator  Comments 

Relevant  operator  comments  were  broken  down  into  the  categories  of  field  of  view, 
grids,  color  sets,  range  data,  update  rate,  test  equipment,  test  design,  and  strategy 
(operator  number  is  included  in  parentheses  after  comments). 

•  Field  of  View-         Narrowing  the  field  of  view  placed  a  premium  on 

maintaining  visual  contact  with  the  targets  (II). 

•  Grids  -  The  "cross-hairs"  effect  aided  steering  effectiveness  (II,  IV). 

Grids  provided  relative  size  information  (II). 

The  3-D  grid  was  more  intuitive  and  effective  than  the  2-D 

grid  (II). 
The  3-D  grid  cluttered  the  display  (IV). 
The  grids  were  not  particularly  helpful  (III). 
Hidden-line  removal  for  grids  could  provide  strong  visual 

range  cues  in  the  immediate  vicinity  of  a  target  by 

showing  the  target  "poking  through"  the  grid  (I,  IV). 
The  black  grids  were  sometimes  confused  with  the  blue 

targets  when  the  display  was  cluttered  (IV). 
The  operator's  need  for  a  grid  was  reduced  by  improvements 

in  his  spatial  perception  (IV). 

•  Color  Sets  -  Color  provided  spatial  visualization  cues,  especially  with 

overlapping  targets  (all). 
The  21 -color  set  provided  significantly  more  information  and 

was  more  effective  than  the  6-color  set  (II,  IV). 
Color  information  was  far  superior  to  gridding  (III). 
Target  course  altitude  slope  information  was  available  from 

the  rate  of  change  of  target  colors  (HI). 
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•  Range  Data  - 


Update  Rate 


Test  Equipment 


Test  Design  - 


A  sliding  color  scale  with  the  vehicle  always  at  scale  center 
could  provide  improved  relative  altitude  cues  (HI). 
Range  information  proved  valuable  if  a  target  was  not 

visible,  was  close,  or  had  been  missed  (all). 
Range  information  was  ignored  if  a  target  was  in  view  (II). 
Range  information  provided  for  smoother  driving  near 

targets  and  eased  hitting  targets  off-center  (II,  IV). 
The  operator  perceived  that  targets  were  "hit"  from  a  greater 

distance  with  range  information  available  (TH). 
Operators  drove  more  conservatively  with  the  slower  update 

rate  (II,  III). 
Update  rate  was  more  important  as  the  range  decreased  (III). 
Vehicle  alignment  with  the  target  trend  was  more  important 

with  the  slower  update  rate  due  to  the  time  lag  in 

update  rate  while  rotating  (TV). 
Operator  workload  increased  with  slower  update  rate  (I). 
The  slow  update  rate  was  not  as  difficult  as  anticipated  (TV). 
The  joystick  was  very  tiring  to  use,  particularly  the  z-axis 

pure-force  control  (III). 

Fine  z-axis  movement  was  difficult  due  to  the  stiff  pure- 
force  control  (II,  IE). 

Axis  movement  with  the  joystick  was  not  compatible  with 
operator  expectations  (I,  HI). 

Presentation  of  all  information  on  one  screen  could  ease 
operator  workload  (all). 

Target  drawing  order  proved  confusing  (I,  II,  III). 

The  display  graphics  can  be  cluttered,  particularly  with  a 
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narrow  field  of  view  (I,  II,  III). 
The  simulated  vehicle  lacked  an  effective  altitude  search 

capability  (I,  III). 
Operator  efficiency  was  reduced  by  the  fixed  2-D  target 

numbers  when  approaching  targets  from  the  side 

(all). 
Improved  heading  perception  was  provided  by  the  fixed  2-D 

target  numbers  (II). 
Strategy  -  Operator  comfort  with  vehicle  rotation  improved  as  testing 

progressed  (I,  III). 
Operators  needed  to  plan  more  than  one  target  in  advance 

and,  when  possible,  kept  them  on  the  display  (II, 

III). 
Rotating  the  vehicle  to  line  up  along  the  target  heading  trend 

reduced  the  time  between  targets  (II,  ni). 
Knowledge  of  the  course  trend  aided  target  identification 

(IV). 
Driving  while  backing  up  was  counterintuitive  (III). 


6.9     Test-Monitor   Observations 

Test-monitor  observations  not  previously  listed  under  operator  comments  were 
broken  down  into  the  categories  of  color  sets,  range  data,  update  rate,  test  equipment,  test 
design,  strategy,  and  tendencies. 

•  Color  Sets  -  Some  colors  were  weak  in  contrast  with  the  background 
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•  Range  Data  - 


Update  Rate 


Test  Equipment 


•  Test  Design  - 


Strategy  - 


dark  gray.  Reddish  Orange  and  Turquoise  were 
most  difficult  for  the  operators  to  discern. 

Range  information  sometimes  handicapped  the  operators  by 
providing  too  much  data,  causing  them  to  ignore  the 
display  visual  information.  Another  detrimental 
effect  was  that  a  control  action  that  caused  the 
range  to  initially  decrease  would  not  necessarily 
result  in  a  hit  As  the  vehicle's  trajectory  passed 
perigee  with  the  target's  center,  the  range  would 
increase.  This  effect  fooled  all  operators  initially, 
particularly  when  moving  the  vehicle  along  more 
than  one  axis  simultaneously. 

The  four-second  update  period  significantly  degraded  the 
operator's  ability  to  mentally  integrate  the  display 
information  and  use  kinetic  depth-effect  cues. 

Stiffness  of  the  joystick  resulted  in  unorthodox  grips  being 
employed,  many  with  both  hands. 

An  artifact  of  the  wireframe  target  design  was  that  the 
simulated  vehicle  could  be  close  to  or  within  the 
target  without  any  portion  of  the  target  being  visible 
on  the  display.  This  is  an  unlikely  scenario  in  the 
field. 

Backing  up  to  keep  the  previous  target  in  view  while  looking 

for  the  next  target  was  an  effective  approach  when 
presented  with  a  limited  field  of  view. 
Heading  information  was  underutilized.  Two  possible  uses 
were  to  measure  the  target  trend  heading  and  align 


58 


the  vehicle  accordingly,  and  to  improve  the  general 

sense  of  spatial  orientation  of  the  operator. 
Operators  adopted  differing  strategies  for  driving.  These 

strategies  were  modified  during  testing  but  did  not 

converge  to  a  single  "correct"  method. 
An  effective  search  technique  when  no  target  was  in  view 

was  critical  in  reducing  course  run  time. 
"Crabbing"  or  twisting  one  direction  while  moving  side  to 

side  in  the  opposite  direction  was  effective  in 

realigning  with  a  new  course  heading  trend  while 

keeping  targets  in  view. 
•  Tendencies  -  Applying  excessive  control  actions  and  failing  to  account  for 

the  delay  caused  by  the  update  rate  was  a  problem, 

especially  when  in  close  proximity  to  a  target. 
Operators  were  hesitant  to  identify  target  numbers  when 

only  a  portion  of  the  number  was  visible  or  the  angle 

of  view  approached  90°. 
As  testing  progressed,  the  operators  became  more  confident 

in  controlling  the  vehicle  in  two  or  three  directions 

simultaneously. 
Operators  occasionally  applied  the  correct  axis-control  order 

in  the  opposite  direction  from  that  intended  This 

was  related  to  the  well-known  outside-in  versus 

inside-out  viewing  problem. 
Operator  average  velocities  increased  as  familiarity  with  the 

testing  improved. 
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Chapter  7 


Summary,  Conclusions  and  Recommendations 


7.1     Summary 

Effective  collection,  computation,  and  presentation  of  undersea  data  to  the  ROV 
pilot  on  a  single  screen  presents  hardware,  software  and  human-factors  problems.  Pilot 
performance  efficiency  through  improved  display  design  is  a  key  human  factors  issue. 
Seven  tests  measured  the  effects  of  an  enhanced  3-D  perspective  display  on  the  simulated 
piloting  of  an  ROV.  The  effects  of  the  ROV  simulation  and  operator  learning  curves  were 
removed  to  directly  compare  performance  changes  due  to  the  various  enhancements. 


7.2     Conclusions 

Display  enhancements  that  improved  operator  performance  were: 

•  widening  the  field  of  view. 

•  providing  a  gridded  screen  reference. 

•  using  vertical  color  schemes  to  cue  depth/altitude  information. 

•  shortening  the  display  update  period. 

The  singularly  most  important  factor  for  the  operator  was  widening  the  field  of 
view.  Providing  the  operator  with  continuous  range  information  to  the  target  center 
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degraded  performance. 

Many  operator  comments  carried  strong  implications  for  system  design. 

•  The  "cross-hairs"  effect  of  a  grid  aided  steering  effectiveness. 

•  Hidden-line  removal  for  grids  could  provide  strong  visual-range  cues  in  the 

immediate  vicinity  of  a  target  by  showing  the  target  "poking  through"  the 
grid. 

•  Color  provided  spatial  visualization  cues  and  altitude  slope  information. 

•  Range  information  proved  valuable  if  a  target  was  not  visible,  was  close,  or  had 

been  missed. 

•  The  update  rate  was  more  important  as  the  range  decreased. 

•  The  joystick  was  very  tiring  to  use.  Fine  z-axis  movement  was  difficult  due  to 

the  stiff  pure-force  control. 

•  Presentation  of  all  information  on  one  screen  could  ease  operator  workload. 

Several  operator  comments  were  not  supported  by  the  test  results. 

•  The  3-D  grid  was  not  more  effective  than  the  2-D  grid.  They  were  nearly 

identical  in  overall  effectiveness. 

•  The  color  schemes  were  not  far  superior  to  the  grids.  The  difference  between  the 

2-D  grid,  the  3-D  grid,  and  the  21 -color  scheme  was  insignificant.  The  6 
color  scheme  was  only  slightly  better. 

•  The  21 -color  set  was  not  more  effective  than  the  6-color  set. 

Several  significant  improvements  to  the  proposed  3-D  perspective  display  are 
available  through  incorporation  of  the  best  features  of  the  display  enhancements  tested.  A 
single  screen  with  as  wide  a  field  of  view  as  possible  and  limited  distortion,  a  cross-hair,  a 
simple  vertical  color  scheme,  target  range  information,  and  a  reasonable  update  period  (two 
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seconds  or  less)  would  improve  ROV  pilot  effectiveness.  More  testing  would  be  necessary 
to  determine  the  best  design  for  a  2-D  or  3-D  grid.  The  key  to  effective  use  of  range 
information  is  the  method  of  employment. 


7.3     Recommendations 

This  thesis  investigated  the  effects  of  individual  display  enhancements  on  operator 
performance.  In  order  to  take  maximum  advantage  of  the  positive  aspects  that  each  of  the 
enhancement  has  to  offer,  several  need  to  be  implemented  simultaneously.  Due  to  the 
potentially  complex  nature  of  the  enhancement  interactions,  it  is  unwise  to  generalize  the 
individual  results  into  group  results.  These  interactions  need  to  be  analyzed  on  a  case 
basis. 

The  presentation  of  both  printed  information  such  as  position  and  velocities  as  well 
as  graphics  on  the  same  screen  was  recommended  by  all  operators.  This  capability  exists 
in  modern  computer  windowing  environments.  The  effectiveness  of  any  proposed  screen 
layout  design  requires  both  qualitative  and  quantitative  assessment 

One  operator  commented  that  a  sliding  color  scale  with  the  vehicle  always  at  scale 
center  could  provide  improved  relative-altitude  cues.  This  suggestion  has  merit  and  bears 
further  investigation. 

Both  operators  and  pilots  expressed  strong  dissatisfaction  with  the  Measurement 
Systems,  Inc.,  joystick  designs  (fourth-axis  control  by  lifting/depressing  the  stick  vertically 
or  pure-force  button).  The  sticks  are  far  too  difficult  to  use  for  extended  periods  due  to 
their  bulk,  stiffness,  and  awkward  incorporation  of  fourth-axis  control.  Catalog  search  has 
yet  to  produce  a  fully  acceptable  alternative,  although  a  different  joystick  model  is  currently 
in  use.  Pilot  preferences  indicate  that  a  modified  video-game  joystick  is  closer  to  their 
concept  of  an  acceptable  control.  Identification  of  design  requirements,  development, 
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testing  to  ensure  effectiveness,  and  procurement  of  a  fully  acceptable  control  is  an  area 
needing  attention. 

Between  testing  sessions,  usually  a  day  or  two  apart,  operators  experienced  a  decay 
of  previously  acquired  skills.  The  driving  of  course  X  prior  to  each  test  appeared  to 
reverse  this  decay.  ROV  piloting  is  not  a  full-time  occupation.  Frequently  months  pass 
between  cruises.  Pilots  also  change  employers,  reducing  the  corporate  piloting  knowledge 
level.  Three  actions  are  recommended  to  reduce  this  loss  of  skills: 

•  Holding  precruise  pilot  discussions  to  reawaken  piloting  skills.  Critical  to  these 

discussions  would  be  the  minutes  of  the  previous  post-cruise  debriefs. 

•  Conduct  of  precruise  pilot  simulator  training.  Construction  of  a  simulator  with 

the  field  equipment  would  be  straightforward  and  all  the  necessary  hardware 
is  already  in  hand.  Pilot  discussions  of  strategies  could  be  demonstrated 
and  tested  prior  to  use  in  the  field.  Added  benefits  of  a  simulator  include 
serving  as  a  test  bed  for  new  display  designs  and  the  ability  to  write  the 
simulator  software  to  incorporate  precruise  system  checks. 

•  Holding  postcruise  debriefs  to  document  problems,  fixes,  and  strategies.  The 

minutes  of  these  debriefs  would  generate  a  valuable  storehouse  of  piloting 
knowledge. 
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APPENDIX  A 


Nomenclature 


Symbol 


Meaning 


Units 


Page 


CD 

4 

(Lj)* 

M 

Ma 

MK 

Mcp 

Mo 

PJ 
T 

P 

e 

K 
C 

Tmax 
<P 

CO 


coefficient  of  drag 

look  direction  matrix  from  point  Pj  to 

point  Pj_|4  in  vehicle  system 
transformation  matrix  from  vehicle 

system  to  world  system 
vehicle  mass 

vehicle  added  mass 

rotation  matrix  for  k  angular  rotation 

rotation  matrix  for  cp  angular  rotation 

rotation  matrix  for  co  angular  rotation 

j-th  data  point 

maximum  thrust 

operator  field  of  view  angular  width 

vehicle  system  rotation  angle  about  z  axis 

world  system  rotation  angle  about  z  axis 

standard  deviation 

linear  drag  (damping)  coefficient 

world  system  rotation  angle  about  y  axis 
world  system  rotation  angle  about  x  axis 


kg/m  or  kg-m 
none 

none 
kg 

kg 

none 

none 

none 

none 

N 

rad 

rad 

rad 

none 

s 

rad 
rad 


38 
37 

37 
38 

38 
37 
37 
37 
35 
38 
35 
26 
36 
51 
38 

36 
36 
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kg  kilograms 

m  meters 

N  newtons 

rad  radians 

s  seconds 
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APPENDIX  B 


Abbreviations 


Symbol 


Meaning 


Page 


ACM  Association  for  Computing  Machinery 

BBN  Bolt,  Beranek,  and  Newman 

CCW  Counterclockwise 

CDR  Commander 

CIE  Commission  Internationale  de  rEclairage 

CRT  Cathode  Ray  Tube 

CW  Clockwise 

DSL  Deep  Submergence  Laboratory 

DV  Dependent  Variable 

EV  Extracted  Value 

FJCC  Fall  Joint  Computer  Congress 

FOV  Field  of  View 

GDM  Graphic  Display  Monitor 

HOOPS  Hierarchical  Object  Oriented  Picture  System 

HSB  Hue,  Saturation,  and  Brightness 

IEEE  Institute  of  Electrical  and  Electronics  Engineers 

IFIP  International  Federation  for  Information  Processing 

ISCC-NBS      Inter-Society  Color  Council  -  National  Bureau  of  Standards 

IV  Independent  Variable 

LCDR  Lieutenant  Commander 


133 
23 
38 
3 
19 
13 
38 
8 
22 
10 

134 

8 

27 

25 

10 

133 

134 
19 
22 

136 
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LCNMT  Learning-Corrected,  Normalized  Mean  Times 

MIPS  Millions  of  Instructions  Per  Second 

MTT  Massachusetts  Institute  of  Technology 

MFLOPS  Millions  of  Floating-Point  Operations  Per  Second 

N/A  Not  Applicable 

NEC  Nippon  Electric  Company 

NTSC  National  Television  System  Committee 

RGB  Red,  Green,  Blue 

ROV  Remotely  Operated  Vehicle 

SF  Standardization  Factor 

SHARPS  Sonic  High  Accuracy  Ranging  and  Positioning  System 

SONAR  Sound  Navigation  and  Ranging 

SSD  Summed  Standard  Deviation 

TI  Texas  Instruments 

UCL  Universal  Color  Language 

USN  United  States  Navy 

VCR  Video  Cassette  Recorder 

VGA  Video  Graphics  Adapter 

WHOI  Woods  Hole  Oceanographic  Institution 

YIQ  Luminance,  In-phase,  and  Quadrature 

2-D  Two-Dimensional 

3-D  Three-Dimensional 


49 
26 

1 
26 
73 
27 
26 

8 

2 
10 
31 

2 
10 
23 
19 

3 
26 
27 

1 
19 

8 

2 
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APPENDIX  C 


Target  and  Grid  Design 


All  seven  tests  used  the  same  25  cubic-wireframe  targets.  Each  target  was  one 
meter  on  a  side  with  the  invisible  x,  y,  and  z  data  point  at  its  center.  Target  faces  were 
parallel  to  the  planes  formed  by  the  world  coordinate  system  axes.  Target  colors  were  blue 
except  for  tests  5  and  6  as  discussed  in  section  5.7.  Each  target  was  numbered  with  white 
numerals  (1-25)  corresponding  to  the  sequential  number  of  the  data  point  at  its  center.  The 
numbers  were  2-D  wireframes  0.5  meters  in  height,  0.3  meters  in  width  (numeral  one  was 
0.1  meters  in  width),  and  drawn  parallel  to  the  XZ  world  coordinate  plane  facing  the  initial 
position  of  the  vehicle.  All  numbers  were  drawn  centered  in  the  target  in  the  x,  y,  and  z 
directions. 

Embedded  in  each  target  was  an  invisible  sphere  of  one-meter  diameter.  This 
sphere  defined  the  "hit"  criterion  for  its  target.  When  the  vehicle,  modeled  as  a  point, 
touched  the  sphere,  the  target  was  hit  This  corresponded  to  closing  the  range  to  the  target 
center  to  0.5  meters  or  less  from  any  direction. 

The  2-D  grid  used  in  test  2  was  composed  of  16  black  squares  arranged  in  four 
rows  of  four.  The  grid  was  drawn  centered,  parallel  to,  and  aligned  with  the  edges  of  the 
display  face  at  a  screen  depth  of  five  meters.  Each  square  was  scaled  so  it  was  one  meter 
on  a  side  at  that  depth.  Figure  C.l  shows  the  operator's  display  of  the  2-D  grid  with  target 
course  X. 

The  3-D  grid  used  in  test  3  was  composed  of  200  black  squares  and  rectangles 
arranged  in  three  panels.  The  center  panel  was  made  up  of  100  squares  in  10  rows  of  10. 
This  grid  was  drawn  centered,  parallel  to,  and  aligned  with  the  edges  of  the  display  face  at 
a  screen  depth  of  five  meters.  Each  square  was  scaled  so  that  it  was  0.25  meter  on  a  side  at 
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that  depth.  The  side  panels  were  each  made  up  of  50  rectangles  in  10  rows  of  five.  Each 
rectangle  was  0.25  meter  in  height  and  one  meter  in  width.  The  side  panels  were  attached 
to  the  center  grid  at  right  angles  to  its  left  and  right  edges.  The  assembled  grid  formed  a 
box  with  no  top  or  bottom  facing  the  operator.  Figure  C.2  shows  the  operator's  display  of 
the  3-D  grid  with  target  course  X. 
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Figure  C.  1       2-D  grid  (operator's  display  with  60°  FOV  and  target  course  X) 


Figure  C.2       3-D  grid  (operator's  display  with  60°  FOV  and  target  course  X) 
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APPENDIX  D 


Data  Point  Courses 


Tables  D.  1  through  D.  19  list  the  x,  y,  and  z  positions  of  the  centers  of  targets  1 
through  25  for  each  of  the  target  courses.  The  method  for  generating  these  x,  y,  and  z 
positions  is  discussed  in  detail  in  section  5.2.  Additionally,  the  shortest  time  difference 
(best  8t )  between  the  centers  of  successive  targets  is  listed  for  courses  A  through  R.  The 
best  5t  between  targets  U  and  V  was  calculated  as  follows: 

•  altitude  distance  from  zu  to  zv  was  calculated,  then  divided  by  the  best  up/down 

speed  (0.25  m/s),  resulting  in  the  best  up/down  time. 

•  the  change  in  vehicle  heading  at  target  U  to  fly  directiy  to  target  V  was  noted. 

•  distance  in  the  horizontal  plane  from  (  xu  ,  yu  )  to  (  xv  ,  yv  )  was  calculated. 

•  this  horizontal  distance  was  decomposed  into  forward  and  lateral  components 

based  on  the  vehicle's  heading  prior  to  reaching  target  U. 

•  the  decomposed  forward  distance  was  divided  by  the  best  forward  speed  (0.5 

m/s). 

•  the  decomposed  lateral  distance  was  divided  by  the  best  lateral  speed  (0.25  m/s). 

•  the  minimum  time  to  travel  the  horizontal  distance  without  heading  change  was 

the  larger  of  the  best  forward  and  lateral  speed  times. 

•  vehicle  heading  change  was  divided  by  the  best  rotational  velocity  (0.157  rad/s). 

•  the  total  horizontal  distance  was  divided  by  the  best  forward  speed  (0.5  m/s). 

•  the  minimum  time  to  travel  the  horizontal  distance  with  heading  change  was  the 

sum  of  the  best  rotational  velocity  time  and  the  best  forward  speed  time. 

•  the  minimum  time  to  travel  the  horizontal  distance  was  the  smaller  of  the  times 

with  and  without  heading  change. 
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•  the  best  time  between  targets  was  the  larger  of  the  best  up/down  and  horizontal 
times. 

Figures  D.l  through  D.18  were  reproduced  from  photographs  of  the  operator's 
initial  display  for  courses  A  through  R  with  a  60°  field  of  view.  Figures  D.19,  D.20,  and 
D.21  were  reproductions  of  the  initial  display  for  course  X  with  a  60°,  30°,  and  15°  field  of 
view  respectively. 
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Table  D.l 


Course  A 


Target 

X 

y 

z 

Best  5t 

1 

75 

5 

50 

N/A 

2 

75.340 

9.971 

50.419 

9.942 

3 

75.683 

14.959 

50.393 

10.000 

4 

75.743 

19.937 

50.862 

9.940 

5 

76.337 

24.897 

51.060 

9.935 

6 

77.204 

29.809 

50.703 

9.959 

7 

78.401 

34.662 

50.821 

9.975 

8 

79.848 

39.447 

50.718 

9.984 

9 

80.992 

44.314 

50.665 

9.980 

10 

82.121 

49.184 

50.707 

10.000 

11 

83.810 

53.888 

50.570 

9.928 

12 

86.077 

58.330 

50.930 

9.894 

13 

88.094 

62.885 

51.366 

9.947 

14 

90.093 

67.411 

52.085 

9.896 

15 

91.666 

72.019 

53.219 

9.704 

16 

92.761 

76.807 

54.158 

9.770 

17 

93.676 

81.488 

55.659 

9.536 

18 

94.076 

86.374 

56.640 

9.742 

19 

94.950 

91.283 

57.016 

9.935 

20 

95.347 

96.261 

57.262 

9.940 

21 

95.595 

101.250 

57.047 

9.984 

22 

95.454 

106.237 

56.713 

9.947 

23 

95.215 

111.159 

55.866 

9.852 

24 

94.726 

115.948 

54.514 

9.612 

25 

94.792 

120.760 

53.159 

9.566 
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Table  D.2 


Course  B 


Target 

X 

y 

z 

Best  5t 

1 

75 

5 

50 

N/A 

2 

75.514 

9.942 

50.561 

9.884 

3 

76.280 

14.801 

51.459 

9.825 

4 

76.464 

19.731 

52.270 

9.800 

5 

76.788 

24.651 

53.100 

9.857 

6 

77.619 

29.517 

53.893 

9.822 

7 

78.714 

34.369 

54.406 

9.933 

8 

80.191 

39.111 

54.981 

9.903 

9 

82.247 

43.558 

55.978 

9.723 

10 

83.945 

48.008 

57.501 

9.500 

11 

85.987 

52.139 

59.440 

9.186 

12 

88.368 

56.088 

61.373 

9.196 

13 

90.182 

60.266 

63.436 

9.039 

14 

91.667 

64.362 

65.889 

9.812 

15 

92.799 

68.246 

68.827 

11.755 

16 

93.692 

72.084 

71.905 

12.311 

17 

94.672 

75.833 

75.064 

12.636 

18 

96.247 

79.760 

77.729 

10.659 

19 

97.585 

83.600 

80.639 

11.639 

20 

98.424 

87.377 

83.806 

12.669 

21 

99.684 

90.714 

87.309 

14.012 

22 

101.492 

93.759 

90.839 

14.118 

23 

102.575 

96.682 

94.748 

15.637 

24 

103.974 

99.466 

98.659 

15.643 

25 

105.626 

101.705 

102.813 

16.618 
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Figure  D.  1        Course  A  (operator's  display  with  60°  FOV) 
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Figure  D.2       Course  B  (operator's  display  with  60°  FOV) 
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Table  D.3 


Course  C 


Target 

X 

y 

z 

Best  8t 

1 

75 

5 

50 

N/A 

2 

75.020 

9.983 

50.415 

9.965 

3 

74.980 

14.977 

50.180 

9.988 

4 

74.925 

19.976 

50.095 

9.999 

5 

75.269 

24.919 

49.429 

9.879 

6 

75.592 

29.763 

48.232 

9.709 

7 

76.443 

34.610 

47.343 

9.786 

8 

77.675 

39.396 

46.586 

9.855 

9 

78.482 

44.192 

45.423 

9.690 

10 

79.342 

48.955 

44.170 

9.680 

11 

80.071 

53.561 

42.368 

9.325 

12 

80.670 

58.230 

40.681 

9.410 

13 

81.492 

63.000 

39.427 

9.672 

14 

82.575 

67.842 

38.810 

9.911 

15 

83.997 

72.636 

38.769 

9.974 

16 

85.192 

77.489 

38.898 

9.986 

17 

86.296 

82.364 

39.015 

9.995 

18 

87.120 

87.270 

38.511 

9.932 

19 

88.257 

92.015 

37.420 

9.738 

20 

89.117 

96.742 

36.035 

9.594 

21 

89.412 

101.410 

34.268 

9.297 

22 

89.760 

106.245 

33.043 

9.695 

23 

89.707 

111.017 

31.552 

9.516 

24 

89.728 

115.746 

29.930 

9.458 

25 

89.851 

120.359 

28.004 

9.226 
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Table  D.4 


Course  D 


Target 

X 

y 

z 

Best  5t 

1 

75 

5 

50 

N/A 

2 

74.372 

9.917 

50.658 

9.835 

3 

74.614 

14.903 

50.928 

9.835 

4 

74.826 

19.671 

52.420 

9.544 

5 

75.283 

24.294 

54.270 

9.278 

6 

76.893 

28.957 

55.080 

9.601 

7 

79.632 

33.111 

55.573 

9.637 

8 

82.281 

37.123 

56.946 

9.615 

9 

85.192 

41.067 

57.934 

9.787 

10 

87.250 

45.189 

59.876 

9.073 

11 

89.045 

48.851 

62.768 

11.570 

12 

91.190 

52.496 

65.436 

10.670 

13 

93.106 

56.769 

67.187 

9.327 

14 

94.735 

61.062 

69.167 

9.169 

15 

95.491 

65.646 

71.015 

9.134 

16 

95.769 

70.151 

73.167 

8.993 

17 

95.051 

74.059 

76.202 

12.139 

18 

94.195 

78.131 

78.975 

11.092 

19 

94.587 

82.593 

81.196 

8.885 

20 

95.963 

86.945 

83.236 

8.962 

21 

96.694 

91.569 

84.993 

9.265 

22 

98.573 

96.153 

85.668 

9.709 

23 

100.225 

100.734 

86.802 

9.734 

24 

101.228 

105.627 

86.583 

9.845 

25 

101.964 

110.571 

86.428 

9.981 
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Figure  D.3       Course  C  (operator's  display  with  60°  FOV) 
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Figure  D.4       Course  D  (operator's  display  with  60°  FOV) 
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Table  D.5 


Course  E 


Target 

X 

y 

z 

Best  5t 

1 

75 

5 

50 

N/A 

2 

74.164 

9.874 

50.737 

9.752 

3 

72.990 

14.566 

52.005 

9.649 

4 

72.104 

19.139 

53.823 

9.301 

5 

70.407 

23.580 

55.370 

9.378 

6 

68.214 

28.060 

55.718 

9.917 

7 

66.758 

32.567 

57.322 

9.357 

8 

64.791 

36.764 

59.197 

9.209 

9 

63.625 

41.499 

60.302 

9.599 

10 

61.257 

45.902 

60.362 

9.663 

11 

59.506 

50.584 

60.486 

9.905 

12 

58.125 

55.374 

60.874 

9.939 

13 

57.941 

60.311 

61.644 

9.591 

14 

58.232 

65.298 

61.853 

9.945 

15 

59.369 

70.150 

62.256 

9.822 

16 

60.703 

74.966 

62.424 

9.986 

17 

63.211 

79.278 

62.770 

9.653 

18 

65.725 

83.598 

62.902 

9.997 

19 

68.894 

87.449 

63.255 

9.849 

20 

71.096 

91.933 

63.470 

9.728 

21 

73.080 

96.326 

64.797 

9.633 

22 

75.684 

100.527 

65.557 

9.796 

23 

77.857 

105.028 

65.683 

9.949 

24 

79.568 

109.626 

64.715 

9.780 

25 

81.847 

113.963 

63.718 

9.723 
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Table  D.6 


Course  F 


Target 

X 

y 

z 

Best  5t 

1 

75 

5 

50 

N/A 

2 

74.090 

9.822 

50.957 

9.651 

3 

72.789 

14.460 

52.299 

9.602 

4 

71.500 

18.826 

54.367 

9.105 

5 

71.318 

23.509 

56.109 

9.130 

6 

71.669 

28.250 

57.657 

9.453 

7 

70.716 

32.909 

59.205 

9.185 

8 

69.282 

37.669 

59.737 

9.897 

9 

67.473 

42.322 

60.019 

9.952 

10 

65.537 

46.929 

60.170 

9.991 

11 

63.072 

51.277 

60.311 

9.927 

12 

61.000 

55.811 

59.923 

9.936 

13 

58.565 

60.063 

58.930 

9.756 

14 

56.794 

64.739 

58.944 

9.864 

15 

53.946 

68.755 

58.072 

9.523 

16 

51.351 

73.023 

57.847 

9.966 

17 

49.014 

77.376 

57.080 

9.867 

18 

45.742 

80.745 

55.364 

9.074 

19 

41.720 

83.431 

54.096 

9.457 

20 

38.238 

87.010 

53.838 

9.831 

21 

34.887 

90.688 

54.334 

9.950 

22 

31.512 

94.286 

53.521 

9.864 

23 

29.020 

98.614 

53.754 

9.792 

24 

27.165 

103.247 

54.071 

9.886 

25 

25.496 

107.871 

53.158 

9.806 
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Figure  D.5       Course  E  (operator's  display  with  60°  FOV) 
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Figure  D.6       Course  F  (operator's  display  with  60°  FOV) 
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Table  D.7 


Course  G 


Target 

X 

y 

z 

Best  5t 

1 

75 

5 

50 

N/A 

2 

77.402 

9.372 

49.658 

9.584 

3 

80.930 

12.001 

47.283 

9.501 

4 

85.722 

13.316 

46.731 

8.949 

5 

90.100 

13.069 

44.328 

9.609 

6 

95.045 

13.750 

44.040 

9.983 

7 

99.803 

12.432 

43.248 

9.344 

8 

104.761 

12.926 

42.829 

9.532 

9 

109.181 

12.034 

44.988 

8.637 

10 

111.078 

10.074 

49.179 

16.763 

11 

110.555 

7.725 

53.562 

17.531 

12 

110.822 

7.113 

58.517 

19.820 

13 

112.792 

8.030 

63.020 

18.011 

14 

115.272 

10.445 

66.628 

14.434 

15 

116.622 

14.175 

69.671 

12.174 

16 

118.163 

16.656 

73.730 

16.233 

17 

117.290 

18.362 

78.348 

18.473 

18 

118.453 

19.258 

83.127 

19.118 

19 

120.460 

18.207 

87.585 

17.829 

20 

123.046 

15.938 

91.212 

14.510 

21 

126.242 

14.134 

94.609 

13.584 

22 

130.759 

13.240 

96.559 

8.860 

23 

135.067 

14.923 

98.456 

9.114 

24 

139.759 

16.585 

98.932 

9.935 

25 

144.021 

18.058 

101.093 

8.962 
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Table  D.8 


Course  H 


Target 

X 

y 

z 

Best  8t 

1 

75 

5 

50 

N/A 

2 

73.035 

9.488 

50.999 

9.010 

3 

71.086 

14.085 

51.257 

9.987 

4 

69.943 

18.154 

53.928 

10.684 

5 

68.558 

21.353 

57.512 

14.338 

6 

65.215 

23.831 

60.286 

11.093 

7 

60.466 

25.325 

60.742 

9.566 

8 

55.907 

26.833 

62.137 

9.558 

9 

52.630 

30.596 

62.454 

9.227 

10 

52.017 

35.556 

62.305 

9.426 

11 

51.968 

40.348 

63.733 

9.281 

12 

50.143 

44.961 

64.353 

9.166 

13 

46.707 

48.593 

64.334 

9.257 

14 

43.809 

52.163 

66.298 

8.822 

15 

40.102 

55.081 

67.954 

9.230 

16 

36.661 

57.474 

70.681 

10.907 

17 

33.390 

57.996 

74.426 

14.981 

18 

31.969 

58.438 

79.200 

19.094 

19 

29.845 

57.500 

83.628 

17.712 

20 

29.339 

57.006 

88.577 

19.799 

21 

26.730 

57.726 

92.782 

16.818 

22 

22.486 

57.380 

95.403 

10.483 

23 

18.256 

58.209 

97.936 

10.135 

24 

14.084 

60.827 

98.797 

9.851 

25 

9.687 

62.529 

100.460 

9.427 
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Figure  D.7       Course  G  (operator's  display  with  60°  FOV) 


Figure  D.8       Course  H  (operator's  display  with  60°  FOV) 
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Table  D.9 


Course  I 


Target 

X 

y 

z 

Best  5t 

1 

75 

5 

50 

N/A 

2 

73.705 

9.827 

49.845 

9.654 

3 

70.962 

13.348 

47.593 

9.011 

4 

69.007 

17.733 

46.196 

9.422 

5 

65.232 

20.735 

44.877 

8.664 

6 

61.594 

24.027 

43.913 

9.805 

7 

59.815 

28.690 

43.604 

9.113 

8 

59.727 

33.487 

45.011 

9.341 

9 

58.587 

38.275 

45.893 

9.711 

10 

55.693 

41.610 

48.238 

9.379 

11 

53.396 

44.963 

51.151 

11.651 

12 

50.001 

45.848 

54.713 

14.250 

13 

47.180 

43.768 

58.279 

14.261 

14 

46.505 

40.286 

61.803 

14.096 

15 

45.997 

37.783 

66.101 

17.195 

16 

45.976 

34.940 

70.214 

16.450 

17 

45.205 

32.056 

74.225 

16.044 

18 

44.872 

28.706 

77.922 

14.787 

19 

42.788 

24.785 

80.222 

9.200 

20 

39.883 

20.716 

80.238 

8.983 

21 

35.754 

18.089 

81.262 

9.452 

22 

31.130 

17.307 

79.527 

8.462 

23 

26.277 

16.223 

80.051 

9.048 

24 

21.766 

15.832 

77.931 

8.483 

25 

17.182 

17.512 

76.854 

9.276 
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Table  D.  10       Course  J 


Target 

X 

y 

z 

Best  8t 

1 

75 

5 

50 

N/A 

2 

74.660 

9.971 

50.419 

9.942 

3 

74.317 

14.959 

50.393 

10.000 

4 

74.257 

19.937 

50.862 

9.940 

5 

73.663 

24.897 

51.060 

9.935 

6 

72.796 

29.809 

50.703 

9.959 

7 

71.599 

34.662 

50.821 

9.975 

8 

70.152 

39.447 

50.718 

9.984 

9 

69.008 

44.314 

50.665 

9.980 

10 

67.879 

49.184 

50.707 

10.000 

11 

66.190 

53.888 

50.570 

9.928 

12 

63.923 

58.330 

50.930 

9.894 

13 

61.906 

62.885 

51.366 

9.947 

14 

59.907 

67.411 

52.085 

9.896 

15 

58.334 

72.019 

53.219 

9.704 

16 

57.239 

76.807 

54.158 

9.770 

17 

56.324 

81.488 

55.659 

9.536 

18 

55.924 

86.374 

56.640 

9.742 

19 

55.050 

91.283 

57.016 

9.935 

20 

54.653 

96.261 

57.262 

9.940 

21 

54.405 

101.250 

57.047 

9.984 

22 

54.546 

106.237 

56.713 

9.947 

23 

54.785 

111.159 

55.866 

9.852 

24 

55.274 

115.948 

54.514 

9.612 

25 

55.208 

120.760 

53.159 

9.566 
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Figure  D.9       Course  I  (operator's  display  with  60°  FOV) 


Figure  D.  10     Course  J  (operator's  display  with  60°  FOV) 


87 


Table  D.  11       Course  K 


Target 

X 

y 

z 

Best  5t 

1 

75 

5 

50 

N/A 

2 

74.486 

9.942 

50.561 

9.884 

3 

73.720 

14.801 

51.459 

9.825 

4 

73.536 

19.731 

52.270 

9.800 

5 

73.212 

24.651 

53.100 

9.857 

6 

72.381 

29.517 

53.893 

9.822 

7 

71.286 

34.369 

54.406 

9.933 

8 

69.809 

39.111 

54.981 

9.903 

9 

67.753 

43.558 

55.978 

9.723 

10 

66.055 

48.008 

57.501 

9.500 

11 

64.013 

52.139 

59.440 

9.186 

12 

61.632 

56.088 

61.373 

9.196 

13 

59.818 

60.266 

63.436 

9.039 

14 

58.333 

64.362 

65.889 

9.812 

15 

57.201 

68.246 

68.827 

11.755 

16 

56.308 

72.084 

71.905 

12.311 

17 

55.328 

75.833 

75.064 

12.636 

18 

53.753 

79.760 

77.729 

10.659 

19 

52.415 

83.600 

80.639 

11.639 

20 

51.576 

87.377 

83.806 

12.669 

21 

50.316 

90.714 

87.309 

14.012 

22 

48.508 

93.759 

90.839 

14.118 

23 

47.425 

96.682 

94.748 

15.637 

24 

46.026 

99.466 

98.659 

15.643 

25 

44.374 

101.705 

102.813 

16.618 
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Table  D.  12       Course  L 


Target 

X 

y 

z 

Best  5t 

1 

75 

5 

50 

N/A 

2 

74.980 

9.983 

50.415 

9.965 

3 

75.020 

14.977 

50.180 

9.988 

4 

75.075 

19.976 

50.095 

9.999 

5 

74.731 

24.919 

49.429 

9.879 

6 

74.408 

29.763 

48.232 

9.709 

7 

73.557 

34.610 

47.343 

9.786 

8 

72.325 

39.396 

46.586 

9.855 

9 

71.518 

44.192 

45.423 

9.690 

10 

70.658 

48.955 

44.170 

9.680 

11 

69.929 

53.561 

42.368 

9.325 

12 

69.330 

58.230 

40.681 

9.410 

13 

68.508 

63.000 

39.427 

9.672 

14 

67.425 

67.842 

38.810 

9.911 

15 

66.003 

72.636 

38.769 

9.974 

16 

64.808 

77.489 

38.898 

9.986 

17 

63.704 

82.364 

39.015 

9.995 

18 

62.880 

87.270 

38.511 

9.932 

19 

61.743 

92.015 

37.420 

9.738 

20 

60.883 

96.742 

36.035 

9.594 

21 

60.588 

101.410 

34.268 

9.297 

22 

60.240 

106.245 

33.043 

9.695 

23 

60.293 

111.017 

31.552 

9.516 

24 

60.272 

115.746 

29.930 

9.458 

25 

60.149 

120.359 

28.004 

9.226 
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Figure  D.  1 1      Course  K  (operator's  display  with  60°  FOV) 


Figure  D.  12     Course  L  (operator's  display  with  60°  FOV) 
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Table  D.  13       Course  M 


Target 

X 

y 

z 

Best  5t 

1 

75 

5 

50 

N/A 

2 

75.628 

9.917 

50.658 

9.835 

3 

75.386 

14.903 

50.928 

9.835 

4 

75.174 

19.671 

52.420 

9.544 

5 

74.717 

24.294 

54.270 

9.278 

6 

73.107 

28.957 

55.080 

9.601 

7 

70.368 

33.111 

55.573 

9.637 

8 

67.719 

37.123 

56.946 

9.615 

9 

64.808 

41.067 

57.934 

9.787 

10 

62.750 

45.189 

59.876 

9.073 

11 

60.955 

48.851 

62.768 

11.570 

12 

58.810 

52.496 

65.436 

10.670 

13 

56.894 

56.769 

67.187 

9.327 

14 

55.265 

61.062 

69.167 

9.169 

15 

54.509 

65.646 

71.015 

9.134 

16 

54.231 

70.151 

73.167 

8.993 

17 

54.949 

74.059 

76.202 

12.139 

18 

55.805 

78.131 

78.975 

11.092 

19 

55.413 

82.593 

81.196 

8.885 

20 

54.037 

86.945 

83.236 

8.962 

21 

53.306 

91.569 

84.993 

9.265 

22 

51.427 

96.153 

85.668 

9.709 

23 

49.775 

100.734 

86.802 

9.734 

24 

48.772 

105.627 

86.583 

9.845 

25 

48.036 

110.571 

86.428 

9.981 
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Table  D.  14       Course  N 


Target 

X 

y 

z 

Best  8t 

1 

75 

5 

50 

N/A 

2 

75.836 

9.874 

50.737 

9.752 

3 

77.010 

14.566 

52.005 

9.649 

4 

77.896 

19.139 

53.823 

9.301 

5 

79.593 

23.580 

55.370 

9.378 

6 

81.786 

28.060 

55.718 

9.917 

7 

83.242 

32.567 

57.322 

9.357 

8 

85.209 

36.764 

59.197 

9.209 

9 

86.375 

41.499 

60.302 

9.599 

10 

88.743 

45.902 

60.362 

9.663 

11 

90.494 

50.584 

60.486 

9.905 

12 

91.875 

55.374 

60.874 

9.939 

13 

92.059 

60.311 

61.644 

9.591 

14 

91.768 

65.298 

61.853 

9.945 

15 

90.631 

70.150 

62.256 

9.822 

16 

89.297 

74.966 

62.424 

9.986 

17 

86.789 

79.278 

62.770 

9.653 

18 

84.275 

83.598 

62.902 

9.997 

19 

81.106 

87.449 

63.255 

9.849 

20 

78.904 

91.933 

63.470 

9.728 

21 

76.920 

96.326 

64.797 

9.633 

22 

74.316 

100.527 

65.557 

9.796 

23 

72.143 

105.028 

65.683 

9.949 

24 

70.432 

109.626 

64.715 

9.780 

25 

68.153 

113.963 

63.718 

9.723 
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Figure  D.  1 3     Course  M  (operator's  display  with  60°  FOV) 


Figure  D.  14     Course  N  (operator's  display  with  60°  FOV) 
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Table  D.  15       Course  O 


Target 

X 

y 

z 

Best  5t 

1 

75 

5 

50 

N/A 

2 

75.910 

9.822 

50.957 

9.651 

3 

77.211 

14.460 

52.299 

9.602 

4 

78.500 

18.826 

54.367 

9.105 

5 

78.682 

23.509 

56.109 

9.130 

6 

78.331 

28.250 

57.657 

9.453 

7 

79.284 

32.909 

59.205 

9.185 

8 

80.718 

37.669 

59.737 

9.897 

9 

82.527 

42.322 

60.019 

9.952 

10 

84.463 

46.929 

60.170 

9.991 

11 

86.928 

51.277 

60.311 

9.927 

12 

89.000 

55.811 

59.923 

9.936 

13 

91.435 

60.063 

58.930 

9.756 

14 

93.206 

64.739 

58.944 

9.864 

15 

96.054 

68.755 

58.072 

9.523 

16 

98.649 

73.023 

57.847 

9.966 

17 

100.986 

77.376 

57.080 

9.867 

18 

104.258 

80.745 

55.364 

9.074 

19 

108.280 

83.431 

54.096 

9.457 

20 

111.762 

87.010 

53.838 

9.831 

21 

115.113 

90.688 

54.334 

9.950 

22 

118.488 

94.286 

53.521 

9.864 

23 

120.980 

98.614 

53.754 

9.792 

24 

122.835 

103.247 

54.071 

9.886 

25 

124.504 

107.871 

53.158 

9.806 

94 


Table  D.  16       Course  P 


Target 

X 

y 

z 

Best  8t 

1 

75 

5 

50 

N/A 

2 

72.598 

9.372 

49.658 

9.584 

3 

69.070 

12.001 

47.283 

9.501 

4 

64.278 

13.316 

46.731 

8.949 

5 

59.900 

13.069 

44.328 

9.609 

6 

54.955 

13.750 

44.040 

9.983 

7 

50.197 

12.432 

43.248 

9.344 

8 

45.239 

12.926 

42.829 

9.532 

9 

40.819 

12.034 

44.988 

8.637 

10 

38.922 

10.074 

49.179 

16.763 

11 

39.445 

7.725 

53.562 

17.531 

12 

39.178 

7.113 

58.517 

19.820 

13 

37.208 

8.030 

63.020 

18.011 

14 

34.728 

10.445 

66.628 

14.434 

15 

33.378 

14.175 

69.671 

12.174 

16 

31.837 

16.656 

73.730 

16.233 

17 

32.710 

18.362 

78.348 

18.473 

18 

31.547 

19.258 

83.127 

19.118 

19 

29.540 

18.207 

87.585 

17.829 

20 

26.954 

15.938 

91.212 

14.510 

21 

23.758 

14.134 

94.609 

13.584 

22 

19.241 

13.240 

96.559 

8.860 

23 

14.933 

14.923 

98.456 

9.114 

24 

10.241 

16.585 

98.932 

9.935 

25 

5.979 

18.058 

101.093 

8.962 
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Figure  D.  15     Course  O  (operator's  display  with  60°  FOV) 


Figure  D.  16     Course  P  (operator's  display  with  60°  FOV) 
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Table  D.  17       Course  Q 


Target 

X 

y 

z 

Best  5t 

1 

75 

5 

50 

N/A 

2 

76.965 

9.488 

50.999 

9.010 

3 

78.914 

14.085 

51.257 

9.987 

4 

80.057 

18.154 

53.928 

10.684 

5 

81.442 

21.353 

57.512 

14.338 

6 

84.785 

23.831 

60.286 

11.093 

7 

89.534 

25.325 

60.742 

9.566 

8 

94.093 

26.833 

62.137 

9.558 

9 

97.370 

30.596 

62.454 

9.227 

10 

97.983 

35.556 

62.305 

9.426 

11 

98.032 

40.348 

63.733 

9.281 

12 

99.857 

44.961 

64.353 

9.166 

13 

103.293 

48.593 

64.334 

9.257 

14 

106.191 

52.163 

66.298 

8.822 

15 

109.898 

55.081 

67.954 

9.230 

16 

113.339 

57.474 

70.681 

10.907 

17 

116.610 

57.996 

74.426 

14.981 

18 

118.031 

58.438 

79.200 

19.094 

19 

120.155 

57.500 

83.628 

17.712 

20 

120.661 

57.006 

88.577 

19.799 

21 

123.270 

57.726 

92.782 

16.818 

22 

127.514 

57.380 

95.403 

10.483 

23 

131.744 

58.209 

97.936 

10.135 

24 

135.916 

60.827 

98.797 

9.851 

25 

140.313 

62.529 

100.460 

9.427 

97 


Table  D.  18       Course  R 


Target 

X 

y 

z 

Best  5t 

1 

75 

5 

50 

N/A 

2 

76.295 

9.827 

49.845 

9.654 

3 

79.038 

13.348 

47.593 

9.011 

4 

80.993 

17.733 

46.196 

9.422 

5 

84.768 

20.735 

44.877 

8.664 

6 

88.406 

24.027 

43.913 

9.805 

7 

90.185 

28.690 

43.604 

9.113 

8 

90.273 

33.487 

45.011 

9.341 

9 

91.413 

38.275 

45.893 

9.711 

10 

94.307 

41.610 

48.238 

9.379 

11 

96.604 

44.963 

51.151 

11.651 

12 

99.999 

45.848 

54.713 

14.250 

13 

102.820 

43.768 

'58.279 

14.261 

14 

103.495 

40.286 

61.803 

14.096 

15 

104.003 

37.783 

66.101 

17.195 

16 

104.024 

34.940 

70.214 

16.450 

17 

104.795 

32.056 

74.225 

16.044 

18 

105.128 

28.706 

77.922 

14.787 

19 

107.212 

24.785 

80.222 

9.200 

20 

110.117 

20.716 

80.238 

8.983 

21 

114.246 

18.089 

81.262 

9.452 

22 

118.870 

17.307 

79.527 

8.462 

23 

123.723 

16.223 

80.051 

9.048 

24 

128.234 

15.832 

77.931 

8.483 

25 

132.818 

17.512 

76.854 

9.276 
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Figure  D.  17     Course  Q  (operator's  display  with  60°  FOV) 


Figure  D.  18     Course  R  (operator's  display  with  60°  FOV) 
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Table  D.  19       Course  X 


Target 

X 

y 

z 

1 

75 

5 

50 

2 

75.415 

9.975 

50.274 

3 

75.804 

14.960 

50.278 

4 

76.682 

19.878 

50.081 

5 

77.443 

24.817 

49.909 

6 

78.169 

29.761 

50.092 

7 

79.052 

34.674 

49.800 

8 

80.541 

39.441 

50.037 

9 

81.479 

44.352 

50.041 

10 

82.073 

49.286 

50.595 

11 

82.721 

54.141 

51.598 

12 

83.762 

58.762 

53.199 

13 

85.367 

62.995 

55.321 

14 

87.501 

66.919 

57.568 

15 

89.307 

71.089 

59.653 

16 

91.327 

75.215 

61.627 

17 

93.395 

79.426 

63.357 

18 

95.436 

83.409 

65.587 

19 

98.061 

87.359 

67.170 

20 

100.328 

91.501 

68.815 

21 

102.772 

95.558 

70.417 

22 

105.399 

99.343 

72.359 

23 

107.766 

103.514 

73.773 

24 

109.633 

107.777 

75.602 

25 

111.801 

111.855 

77.516 
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Figure  D.  19     Course  X  (operator's  display  with  60°  FOV) 


Figure  D.20     Course  X  (operator's  display  with  30°  FOV) 
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Figure  D.21      Course  X  (operator's  display  with  15°  FOV) 
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APPENDIX  E 


Raw  Data 


Tables  E.l  through  E.12  list  the  data  for  each  of  the  four  operators  over  all  seven 
tests.  The  tabulated  values  are  the  time  differences  between  successive  targets  in  seconds. 
Value  1  is  the  time  difference  between  hitting  targets  1  and  2,  value  2  for  targets  2  and  3, 
etc.  The  total  for  each  column  is  indicated  at  the  bottom  of  the  column.  For  all  tests  the 
course  letter  is  specified  at  the  top  of  its  respective  column.  For  Test  1  the  operator's  field 
of  view  is  also  shown. 
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Table  E.  1  Operator  I,  Test  1 


60°FOV 

30°  FOV 

15°  FOV 

A 

D 

G 

B 

E 

H 

C 

F 

I 

1 

16 

16 

13 

55 

53 

35 

12 

53 

42 

2 

10 

17 

39 

50 

59 

38 

39 

90 

49 

3 

14 

15 

16 

33 

67 

101 

28 

74 

29 

4 

15 

37 

28 

40 

78 

59 

33 

31 

45 

5 

15 

22 

32 

36 

79 

48 

35 

82 

79 

6 

15 

22 

22 

33 

55 

34 

75 

50 

43 

7 

14 

15 

14 

33 

27 

23 

72 

96 

32 

8 

11 

17 

28 

194 

25 

39 

83 

60 

33 

9 

11 

27 

129 

48 

11 

11 

76 

21 

219 

10 

11 

35 

88 

64 

32 

36 

92 

33 

27 

11 

18 

29 

180 

31 

82 

22 

89 

43 

189 

12 

17 

24 

67 

116 

37 

41 

51 

31 

221 

13 

15 

18 

34 

75 

14 

106 

63 

19 

46 

14 

14 

20 

18 

39 

68 

44 

69 

61 

47 

15 

46 

19 

25 

73 

69 

94 

32 

26 

59 

16 

36 

49 

44 

77 

157 

52 

43 

93 

109 

17 

16 

21 

78 

50 

38 

195 

56 

33 

114 

18 

15 

31 

93 

24 

14 

48 

42 

22 

25 

19 

13 

18 

104 

51 

21 

46 

89 

25 

41 

20 

12 

16 

38 

85 

47 

60 

35 

21 

83 

21 

11 

21 

51 

30 

17 

57 

40 

24 

22 

22 

11 

16 

38 

47 

11 

26 

34 

19 

21 

23 

10 

18 

27 

152 

41 

42 

33 

16 

28 

24 

66 

12 

12 

52 

18 

60 

32 

13 

138 

Total 

432 

535 

1218 

1488 

1120 

1317 

1253 

1036 

1741 

104 


Table  E.2  Operator  I  -  Tests  2,  3,  and  4 


Test  2 

Test  3 

Test  4 

J 

M 

P 

K 

N 

Q 

A 

E 

I 

1 

16 

16 

41 

15 

13 

20 

18 

19 

16 

2 

16 

11 

36 

14 

17 

17 

13 

24 

26 

3 

11 

15 

21 

10 

15 

20 

11 

16 

18 

4 

17 

29 

28 

14 

22 

28 

14 

23 

28 

5 

24 

46 

19 

18 

18 

36 

15 

13 

36 

6 

19 

12 

26 

16 

55 

91 

32 

16 

13 

7 

29 

17 

14 

38 

22 

27 

16 

25 

19 

8 

13 

16 

53 

18 

13 

31 

11 

12 

47 

9 

14 

22 

37 

14 

15 

17 

10 

20 

16 

10 

24 

37 

93 

15 

12 

50 

11 

17 

17 

11 

28 

18 

105 

13 

13 

16 

15 

13 

53 

12 

24 

21 

65 

11 

10 

23 

18 

11 

144 

13 

12 

25 

88 

13 

12 

40 

24 

20 

19 

14 

13 

23 

27 

13 

11 

21 

11 

19 

29 

15 

18 

26 

52 

47 

16 

22 

12 

11 

33 

16 

25 

38 

47 

16 

12 

48 

12 

15 

49 

17 

18 

34 

47 

22 

18 

30 

13 

13 

28 

18 

27 

17 

69 

16 

18 

69 

13 

12 

20 

19 

22 

26 

102 

18 

14 

70 

14 

14 

22 

20 

42 

20 

32 

25 

15 

36 

11 

15 

30 

21 

19 

20 

19 

20 

11 

37 

14 

10 

17 

22 

25 

22 

32 

39 

9 

32 

16 

12 

10 

23 

18 

12 

25 

22 

16 

12 

12 

21 

14 

24 

18 

10 

30 

23 

10 

14 

11 

16 

23 

Total 

492 

533 

1108 

470 

387 

807 

347 

387 

727 

105 


Table  E.3         Operator  I  -  Tests  5,  6,  and  7 


Test  5 

Test  6 

Test  7 

B 

F 

G 

L 

O 

R 

C 

D 

H 

1 

11 

12 

50 

10 

17 

29 

10 

25 

56 

2 

48 

19 

39 

10 

11 

31 

10 

22 

20 

3 

18 

17 

14 

11 

32 

24 

10 

16 

22 

4 

15 

13 

25 

9 

15 

23 

18 

29 

38 

5 

14 

13 

25 

12 

12 

18 

19 

31 

47 

6 

17 

16 

20 

12 

21 

67 

22 

28 

13 

7 

17 

16 

17 

12 

14 

25 

19 

35 

18 

8 

10 

10 

20 

13 

14 

30 

21 

29 

47 

9 

10 

10 

67 

22 

14 

24 

24 

27 

15 

10 

28 

19 

62 

18 

17 

22 

48 

26 

18 

11 

39 

18 

38 

18 

18 

32 

19 

37 

37 

12 

29 

14 

49 

20 

16 

85 

20 

27 

17 

13 

29 

18 

91 

15 

16 

50 

18 

36 

37 

14 

23 

15 

39 

12 

33 

25 

12 

27 

34 

15 

22 

10 

33 

13 

21 

30 

10 

20 

22 

16 

23 

31 

31 

12 

13 

25 

10 

24 

55 

17 

27 

13 

41 

13 

33 

24 

14 

37 

37 

18 

22 

26 

86 

23 

21 

32 

51 

18 

60 

19 

26 

13 

45 

14 

14 

17 

28 

32 

54 

20 

21 

22 

28 

11 

14 

44 

16 

24 

25 

21 

26 

21 

16 

11 

18 

19 

15 

33 

27 

22 

55 

14 

20 

10 

23 

17 

16 

15 

42 

23 

28 

12 

18 

11 

13 

17 

17 

10 

20 

24 

24 

10 

14 

15 

10 

16 

15 

11 

32 

Total 

582 

382 

888 

327 

430 

726 

462 

619 

793 

106 


Table  E.4         Operator  II  -  Test  1 


60°FOV 

30°  FOV 

15°  FOV 

J 

M 

P 

K 

N 

Q 

L 

O 

R 

1 

14 

15 

18 

22 

39 

40 

27 

32 

25 

2 

15 

32 

88 

15 

31 

49 

22 

44 

68 

3 

12 

44 

13 

43 

76 

38 

51 

28 

61 

4 

10 

18 

53 

20 

45 

38 

16 

31 

26 

5 

12 

57 

12 

25 

32 

203 

33 

38 

41 

6 

10 

35 

97 

15 

53 

52 

67 

124 

19 

7 

36 

23 

20 

76 

29 

35 

71 

29 

47 

8 

66 

21 

75 

15 

30 

52 

78 

92 

88 

9 

19 

15 

239 

24 

36 

74 

32 

37 

43 

10 

23 

107 

77 

16 

33 

23 

32 

115 

77 

11 

19 

47 

51 

116 

62 

16 

34 

48 

87 

12 

24 

44 

69 

37 

24 

93 

36 

80 

115 

13 

31 

20 

55 

30 

26 

185 

25 

59 

98 

14 

63 

30 

54 

59 

77 

27 

41 

99 

94 

15 

19 

21 

56 

30 

13 

43 

36 

44 

86 

16 

25 

27 

38 

110 

49 

32 

20 

59 

47 

17 

14 

20 

259 

87 

16 

63 

69 

245 

347 

18 

82 

20 

132 

31 

20 

59 

17 

56 

130 

19 

42 

16 

87 

61 

24 

43 

34 

130 

220 

20 

27 

13 

66 

44 

35 

63 

34 

263 

133 

21 

11 

12 

85 

60 

43 

57 

274 

43 

113 

22 

19 

27 

34 

52 

63 

30 

31 

57 

97 

23 

29 

19 

51 

42 

17 

29 

85 

71 

160 

24 

87 

10 

34 

70 

86 

19 

125 

72 

61 

Total 

709 

693 

1763 

1100 

959 

1363 

1290 

1896 

2283 

107 


Table  E.5  Operator  II  -  Tests  2,  3,  and  4 


Test  2 

Test  3 

Test  4 

A 

D 

G 

B 

E 

H 

J 

N 

R 

1 

20 

16 

21 

12 

25 

30 

12 

20 

30 

2 

12 

12 

28 

22 

25 

17 

12 

13 

30 

3 

14 

19 

23 

14 

50 

17 

10 

14 

32 

4 

13 

55 

26 

14 

36 

30 

15 

22 

40 

5 

25 

34 

21 

16 

23 

43 

14 

16 

18 

6 

20 

22 

26 

22 

16 

82 

12 

23 

18 

7 

21 

25 

17 

22 

22 

39 

11 

23 

11 

8 

14 

18 

17 

33 

23 

59 

10 

17 

15 

9 

16 

25 

213 

32 

24 

17 

11 

17 

37 

10 

19 

33 

45 

33 

13 

23 

10 

19 

34 

11 

22 

35 

82 

28 

18 

18 

10 

16 

72 

12 

26 

22 

150 

54 

15 

17 

13 

12 

56 

13 

20 

125 

53 

35 

15 

41 

18 

10 

30 

14 

25 

32 

76 

33 

80 

22 

15 

12 

71 

15 

20 

21 

45 

30 

31 

27 

14 

17 

79 

16 

17 

27 

28 

38 

31 

48 

17 

46 

30 

17 

13 

23 

78 

36 

30 

44 

17 

16 

35 

18 

15 

32 

49 

33 

30 

54 

17 

22 

23 

19 

10 

25 

99 

25 

40 

66 

18 

12 

16 

20 

10 

20 

29 

32 

28 

64 

18 

15 

24 

21 

11 

17 

51 

58 

45 

74 

20 

14 

33 

22 

18 

22 

60 

30 

49 

65 

25 

14 

20 

23 

17 

16 

54 

32 

26 

15 

27 

9 

23 

24 

35 

24 

39 

28 

36 

48 

17 

15 

48 

Total 

433 

700 

1330 

712 

731 

960 

363 

414 

825 

108 


Table  E.6         Operator  II  -  Tests  5,  6,  and  7 


Test  5 

Test  6 

Test  7 

K 

O 

P 

C 

F 

I 

L 

M 

Q 

1 

11 

22 

50 

13 

12 

31 

20 

25 

19 

2 

10 

20 

37 

27 

14 

48 

17 

20 

28 

3 

12 

25 

25 

30 

21 

26 

17 

24 

41 

4 

13 

23 

25 

21 

10 

32 

53 

20 

40 

5 

11 

22 

25 

17 

26 

28 

28 

21 

50 

6 

12 

22 

13 

27 

20 

22 

20 

58 

38 

7 

10 

21 

14 

17 

14 

17 

23 

26 

28 

8 

11 

25 

18 

16 

13 

20 

45 

28 

21 

9 

16 

15 

37 

25 

19 

22 

33 

22 

27 

10 

18 

13 

52 

22 

19 

23 

37 

23 

22 

11 

18 

14 

54 

19 

18 

74 

19 

23 

37 

12 

21 

43 

32 

19 

22 

136 

32 

23 

14 

13 

25 

25 

39 

16 

12 

72 

22 

25 

27 

14 

26 

18 

43 

23 

18 

87 

14 

23 

28 

15 

29 

15 

36 

27 

17 

23 

26 

26 

53 

16 

30 

20 

42 

16 

13 

40 

53 

38 

76 

17 

20 

23 

35 

16 

25 

41 

25 

37 

181 

18 

19 

28 

54 

16 

21 

38 

24 

32 

50 

19 

25 

33 

32 

13 

12 

22 

23 

20 

46 

20 

21 

19 

28 

15 

19 

28 

22 

19 

41 

21 

30 

14 

49 

17 

21 

45 

26 

17 

37 

22 

80 

11 

42 

16 

11 

94 

16 

20 

44 

23 

37 

12 

19 

19 

16 

32 

19 

27 

42 

24 

31 

12 

28 

19 

38 

51 

38 

12 

16 

Total 

536 

495 

829 

466 

431 

1052 

652 

609 

1006 

109 


Table  E.7         Operator  III  -  Test  1 


60°FOV 

30°  FOV 

15°  FOV 

B 

E 

H 

C 

F 

I 

A 

D 

G 

1 

10 

29 

21 

10 

54 

31 

14 

10 

222 

2 

12 

78 

16 

11 

40 

49 

11 

10 

398 

3 

11 

39 

49 

13 

53 

65 

20 

185 

28 

4 

10 

29 

39 

64 

45 

88 

55 

153 

287 

5 

11 

40 

32 

52 

61 

47 

24 

33 

46 

6 

11 

34 

18 

69 

47 

39 

18 

30 

29 

7 

10 

28 

23 

54 

35 

113 

29 

46 

32 

8 

10 

54 

27 

76 

44 

170 

13 

19 

35 

9 

10 

14 

18 

59 

47 

149 

9 

26 

124 

10 

36 

11 

27 

67 

69 

109 

11 

31 

182 

11 

9 

11 

17 

101 

45 

203 

14 

116 

68 

12 

15 

12 

10 

70 

50 

95 

17 

28 

82 

13 

14 

110 

19 

76 

86 

61 

53 

24 

101 

14 

16 

33 

34 

63 

99 

90 

24 

20 

71 

15 

68 

17 

20 

48 

47 

93 

28 

21 

42 

16 

57 

12 

76 

57 

43 

32 

41 

37 

72 

17 

17 

18 

25 

55 

59 

30 

17 

37 

37 

18 

15 

42 

86 

65 

61 

120 

11 

158 

102 

19 

62 

17 

256 

70 

40 

17 

10 

32 

230 

20 

42 

29 

101 

69 

46 

17 

9 

34 

397 

21 

19 

30 

45 

73 

53 

21 

10 

21 

127 

22 

142 

28 

41 

85 

29 

17 

13 

62 

26 

23 

99 

20 

46 

67 

62 

26 

14 

9 

23 

24 

66 

38 

117 

43 

31 

65 

13 

10 

44 

Total 

772 

773 

1163 

1417 

1246 

1747 

478 

1152 

2805 

110 


Table  E.8         Operator  in  -  Tests  2,  3,  and  4 


Test  2 

Test  3 

Test  4 

K 

N 

Q 

L 

O 

R 

B 

F 

G 

1 

14 

28 

26 

10 

16 

13 

19 

15 

63 

2 

30 

18 

18 

10 

18 

20 

36 

12 

35 

3 

33 

34 

28 

33 

21 

11 

26 

16 

79 

4 

34 

18 

51 

28 

32 

22 

12 

38 

25 

5 

20 

19 

29 

102 

13 

15 

21 

38 

26 

6 

29 

21 

23 

44 

40 

11 

58 

24 

29 

7 

20 

17 

72 

12 

17 

14 

12 

13 

18 

8 

19 

31 

43 

17 

20 

21 

21 

17 

17 

9 

13 

28 

17 

21 

79 

14 

16 

24 

73 

10 

19 

15 

76 

22 

64 

23 

26 

10 

110 

11 

36 

17 

36 

20 

22 

54 

20 

13 

94 

12 

28 

22 

15 

15 

12 

32 

20 

14 

45 

13 

68 

14 

65 

58 

34 

26 

25 

13 

55 

14 

32 

20 

29 

10 

16 

70 

17 

15 

33 

15 

20 

23 

30 

10 

16 

40 

19 

12 

37 

16 

67 

31 

47 

11 

43 

44 

26 

16 

55 

17 

27 

26 

38 

12 

22 

43 

18 

23 

37 

18 

23 

21 

51 

33 

64 

11 

19 

22 

74 

19 

24 

25 

39 

15 

30 

21 

34 

16 

54 

20 

29 

23 

47 

97 

12 

54 

32 

13 

29 

21 

33 

15 

41 

21 

30 

28 

36 

17 

19 

22 

46 

11 

31 

16 

20 

20 

30 

17 

23 

23 

40 

18 

62 

9 

11 

15 

40 

11 

14 

24 

36 

12 

21 

9 

19 

55 

36 

13 

46 

Total 

740 

507 

935 

635 

671 

677 

619 

422 

1090 

111 


Table  E.9         Operator  in  -  Tests  5,  6,  and  7 


Test  5 

Test  6 

Test  7 

C 

D 

H 

J 

M 

P 

A 

E 

I 

1 

10 

21 

15 

11 

26 

11 

16 

24 

20 

2 

10 

10 

10 

9 

15 

34 

19 

23 

62 

3 

10 

20 

23 

43 

22 

19 

21 

51 

59 

4 

13 

58 

33 

16 

37 

23 

20 

23 

35 

5 

21 

14 

23 

12 

69 

15 

12 

35 

46 

6 

46 

14 

20 

11 

24 

45 

21 

27 

20 

7 

17 

21 

15 

10 

18 

19 

24 

58 

41 

8 

44 

17 

30 

10 

17 

36 

15 

44 

38 

9 

41 

21 

16 

11 

13 

49 

24 

25 

25 

10 

54 

14 

36 

10 

69 

72 

26 

19 

32 

11 

19 

26 

28 

14 

28 

83 

21 

20 

108 

12 

48 

21 

17 

10 

35 

71 

28 

30 

98 

13 

24 

22 

22 

9 

31 

68 

56 

19 

66 

14 

20 

22 

14 

12 

30 

75 

22 

18 

27 

15 

17 

23 

21 

10 

26 

63 

19 

17 

29 

16 

15 

24 

33 

18 

31 

50 

30 

19 

58 

17 

14 

21 

51 

16 

24 

58 

35 

13 

51 

18 

15 

21 

42 

11 

30 

48 

11 

32 

67 

19 

24 

20 

52 

57 

32 

49 

24 

20 

35 

20 

10 

38 

39 

15 

35 

51 

26 

37 

20 

21 

12 

11 

48 

14 

25 

38 

17 

104 

35 

22 

10 

10 

52 

16 

23 

65 

26 

36 

17 

23 

25 

10 

22 

9 

19 

26 

50 

28 

19 

24 

23 

10 

18 

16 

17 

14 

24 

28 

26 

Total 

542 

489 

680 

370 

696 

1082 

587 

750 

1034 

112 


Table  E.  1 0       Operator  IV  -  Test  1 


60°  FOV 

30°  FOV 

15°  FOV 

K 

N 

Q 

L 

O 

R 

J 

M 

P 

1 

38 

23 

25 

40 

51 

79 

76 

46 

35 

2 

37 

46 

32 

44 

46 

77 

86 

30 

51 

3 

30 

27 

51 

38 

69 

51 

67 

132 

48 

4 

49 

25 

38 

62 

39 

61 

48 

61 

110 

5 

54 

41 

38 

70 

39 

83 

64 

297 

54 

6 

39 

49 

20 

39 

24 

19 

53 

50 

277 

7 

67 

25 

23 

43 

27 

61 

43 

73 

56 

8 

51 

48 

33 

79 

18 

48 

30 

24 

316 

9 

28 

42 

24 

26 

15 

183 

21 

51 

140 

10 

25 

36 

22 

36 

14 

61 

49 

52 

310 

11 

22 

49 

20 

40 

11 

253 

46 

34 

219 

12 

23 

25 

20 

25 

29 

270 

31 

51 

152 

13 

33 

15 

34 

26 

37 

171 

104 

202 

179 

14 

42 

31 

31 

41 

42 

282 

135 

216 

258 

15 

45 

19 

35 

22 

38 

119 

43 

45 

90 

16 

32 

21 

56 

17 

14 

141 

97 

61 

127 

17 

30 

24 

46 

20 

46 

95 

40 

54 

96 

18 

49 

18 

74 

33 

48 

61 

99 

176 

88 

19 

39 

21 

33 

29 

15 

20 

127 

58 

104 

20 

66 

22 

145 

26 

25 

69 

121 

69 

64 

21 

75 

35 

28 

20 

171 

79 

46 

14 

44 

22 

44 

47 

40 

19 

53 

66 

32 

40 

63 

23 

36 

31 

66 

38 

35 

100 

57 

41 

121 

24 

45 

14 

46 

47 

39 

20 

60 

30 

79 

Total 

999 

734 

980 

880 

945 

2469 

1575 

1907 

3081 

113 


Table  E.  1 1        Operator  IV  -  Tests  2,  3,  and  4 


Test  2 

Test  3 

Test  4 

B 

E 

H 

C 

F 

I 

K 

O 

P 

1 

11 

17 

19 

11 

15 

21 

12 

11 

29 

2 

20 

22 

13 

12 

38 

23 

23 

17 

41 

3 

16 

21 

40 

10 

21 

18 

18 

24 

18 

4 

17 

27 

41 

12 

16 

53 

35 

26 

23 

5 

17 

37 

54 

17 

12 

30 

21 

22 

16 

6 

14 

21 

21 

15 

16 

22 

16 

17 

15 

7 

13 

17 

28 

13 

19 

17 

17 

13 

18 

8 

22 

22 

19 

22 

19 

14 

11 

30 

22 

9 

20 

19 

19 

18 

18 

22 

11 

18 

46 

10 

35 

22 

16 

55 

16 

24 

13 

10 

56 

11 

36 

26 

16 

19 

18 

109 

15 

43 

25 

12 

18 

13 

58 

17 

31 

77 

16 

16 

152 

13 

14 

10 

20 

14 

16 

77 

14 

14 

48 

14 

17 

16 

23 

11 

21 

109 

14 

17 

22 

15 

15 

12 

32 

13 

36 

29 

18 

18 

35 

16 

17 

16 

60 

14 

27 

36 

23 

13 

103 

17 

53 

12 

59 

12 

16 

26 

19 

16 

76 

18 

18 

20 

66 

14 

11 

23 

18 

13 

84 

19 

52 

18 

133 

12 

14 

11 

21 

11 

59 

20 

29 

18 

36 

15 

11 

19 

22 

13 

31 

21 

28 

20 

42 

11 

11 

20 

67 

14 

16 

22 

68 

17 

31 

12 

18 

17 

29 

14 

37 

23 

22 

11 

22 

10 

15 

13 

53 

10 

11 

24 

17 

11 

17 

9 

10 

17 

43 

10 

28 

Total 

589 

445 

885 

368 

445 

827 

549 

410 

1011 
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Table  E.  1 2       Operator  IV  -  Tests  5,  6,  and  7 


Test  5 

Test  6 

Test  7 

L 

M 

Q 

A 

D 

G 

J 

N 

R 

1 

11 

17 

17 

19 

15 

25 

12 

32 

12 

2 

12 

11 

11 

29 

17 

23 

14 

34 

35 

3 

10 

17 

20 

20 

13 

19 

23 

28 

71 

4 

14 

13 

24 

13 

60 

28 

17 

24 

24 

5 

13 

11 

21 

12 

30 

16 

13 

23 

18 

6 

12 

11 

16 

13 

12 

15 

13 

22 

40 

7 

19 

13 

58 

16 

13 

18 

14 

24 

25 

8 

13 

11 

16 

13 

14 

24 

14 

25 

29 

9 

12 

12 

15 

22 

19 

45 

15 

14 

28 

10 

17 

17 

38 

20 

29 

84 

12 

20 

23 

11 

13 

20 

14 

10 

20 

43 

21 

16 

60 

12 

14 

17 

15 

11 

19 

61 

26 

19 

61 

13 

39 

13 

17 

10 

25 

58 

28 

12 

53 

14 

11 

22 

19 

12 

22 

99 

25 

20 

172 

15 

10 

18 

124 

11 

13 

51 

20 

14 

42 

16 

10 

19 

32 

10 

75 

40 

35 

23 

44 

17 

52 

17 

75 

11 

29 

92 

24 

17 

31 

18 

14 

27 

91 

9 

21 

72 

13 

18 

22 

19 

12 

14 

45 

11 

26 

73 

13 

19 

11 

20 

12 

16 

52 

20 

20 

23 

25 

17 

26 

21 

15 

9 

35 

17 

20 

16 

32 

23 

26 

22 

12 

10 

25 

13 

17 

22 

18 

13 

19 

23 

10 

10 

14 

10 

12 

15 

14 

20 

24 

24 

9 

10 

23 

48 

11 

28 

26 

14 

18 

Total 

366 

355 

817 

380 

552 

990 

467 

491 

914 
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APPENDIX  F 


Normalized  Data 


Tables  F.l  through  F.12  list  the  data  for  each  of  the  four  operators  over  all  seven 
tests.  The  tabulated  values  are  normalized  values  from  the  associated  tables  in  Appendix  E. 
The  data  were  normalized  to  remove  effects  properly  attributed  to  variations  in  the  level  of 
course  difficulty  related  to  the  Jason  vehicle  simulation  model.  The  total,  mean  value,  and 
standard  deviation  (o)  for  each  column  are  indicated  at  the  bottom  of  the  column.  For  all 
tests  the  course  letter  is  specified  at  the  top  of  its  respective  column.  For  Test  1  the 
operator's  field  of  view  is  also  shown. 

The  normalization  procedure  divided  each  time  difference  by  the  best  time  between 
the  associated  targets  from  Appendix  D  to  arrive  at  the  normalized  value  (dimensionless). 
Because  the  logged  times  represented  the  most  recently  passed  whole  second,  while  the 
best  times  between  targets  were  calculated  to  a  greater  degree  of  precision,  it  was  possible 
to  achieve  a  normalized  value  of  0.9. 
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Table  F.  1  Operator  I  -  Test  1 


60°FOV 

30°  FOV 

15°  FOV 

A 

D 

G 

B 

E 

H 

C 

F 

I 

1 

1.6 

1.6 

1.4 

5.6 

5.4 

3.9 

1.2 

5.5 

4.4 

2 

1.0 

1.7 

4.1 

5.1 

6.1 

3.8 

3.9 

9.4 

5.4 

3 

1.4 

1.6 

1.8 

3.4 

7.2 

9.5 

2.8 

8.1 

3.1 

4 

1.5 

4.0 

2.9 

4.1 

8.3 

4.1 

3.3 

3.4 

5.2 

5 

1.5 

2.3 

3.2 

3.7 

8.0 

4.3 

3.6 

8.7 

8.1 

6 

1.5 

2.3 

2.4 

3.3 

5.9 

3.6 

7.7 

5.4 

4.7 

7 

1.4 

1.6 

1.5 

3.3 

2.9 

2.4 

7.3 

9.7 

3.4 

8 

1.1 

1.7 

3.2 

20.0 

2.6 

4.2 

8.6 

6.0 

3.4 

9 

1.1 

3.0 

7.7 

5.1 

1.1 

1.2 

7.9 

2.1 

23.4 

10 

1.1 

3.0 

5.0 

7.0 

3.2 

3.9 

9.9 

3.3 

2.3 

11 

1.8 

2.7 

9.1 

3.4 

8.3 

2.4 

9.5 

4.3 

13.3 

12 

1.7 

2.6 

3.7 

12.8 

3.9 

4.4 

5.3 

3.2 

15.5 

13 

1.5 

2.0 

2.4 

7.6 

1.4 

12.0 

6.4 

1.9 

3.3 

14 

1.4 

2.2 

1.5 

3.3 

6.9 

4.8 

6.9 

6.4 

2.7 

15 

4.7 

2.1 

1.5 

5.9 

6.9 

8.6 

3.2 

2.6 

3.6 

16 

3.8 

4.0 

2.4 

6.1 

16.3 

3.5 

4.3 

9.4 

6.8 

17 

1.6 

1.9 

4.1 

4.7 

3.8 

10.2 

5.6 

3.6 

7.7 

18 

1.5 

3.5 

5.2 

2.1 

1.4 

2.7 

4.3 

2.3 

2.7 

19 

1.3 

2.0 

7.2 

4.0 

2.2 

2.3 

9.3 

2.5 

4.6 

20 

1.2 

1.7 

2.8 

6.1 

4.9 

3.6 

3.8 

2.1 

8.8 

21 

1.1 

2.2 

5.8 

2.1 

1.7 

5.4 

4.1 

2.4 

2.6 

22 

1.1 

1.6 

4.2 

3.0 

1.1 

2.6 

3.6 

1.9 

2.3 

23 

1.0 

1.8 

2.7 

9.7 

4.2 

4.3 

3.5 

1.6 

3.3 

24 

6.9 

1.2 

1.3 

3.1 

1.9 

6.4 

3.5 

1.3 

14.9 

Total 

44.0 

54.3 

86.9 

134.4 

115.6 

114.0 

129.3 

107.5 

155.3 

Mean 

1.8 

2.3 

3.6 

5.6 

4.8 

4.8 

5.4 

4.5 

6.5 

o 

1.4 

0.7 

2.1 

3.8 

3.4 

2.7 

2.4 

2.7 

5.2 
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Table  F.2         Operator  I  -  Tests  2,  3,  and  4 


Test  2 

Test  3 

Test  4 

J 

M 

P 

K 

N 

Q 

A 

E 

I 

1 

1.6 

1.6 

4.3 

1.5 

1.3 

2.2 

1.8 

1.9 

1.7 

2 

1.6 

1.1 

3.8 

1.4 

1.8 

1.7 

1.3 

2.5 

2.9 

3 

1.1 

1.6 

2.3 

1.0 

1.6 

1.9 

1.1 

1.7 

1.9 

4 

1.7 

3.1 

2.9 

1.4 

2.3 

2.0 

1.4 

2.5 

3.2 

5 

2.4 

4.8 

1.9 

1.8 

1.8 

3.2 

1.5 

1.3 

3.7 

6 

1.9 

1.2 

2.8 

1.6 

5.9 

9.5 

3.2 

1.7 

1.4 

7 

2.9 

1.8 

1.5 

3.8 

2.4 

2.8 

1.6 

2.7 

2.0 

8 

1.3 

1.6 

6.1 

1.9 

1.4 

3.4 

1.1 

1.3 

4.8 

9 

1.4 

2.4 

2.2 

1.5 

1.6 

1.8 

1.0 

2.1 

1.7 

10 

2.4 

3.2 

5.3 

1.6 

1.2 

5.4 

1.1 

1.7 

1.5 

11 

2.8 

1.7 

5.3 

1.4 

1.3 

1.7 

1.5 

1.3 

3.7 

12 

2.4 

2.3 

3.6 

1.2 

1.0 

2.5 

1.8 

1.1 

10.1 

13 

1.2 

2.7 

6.1 

1.3 

1.2 

4.5 

2.4 

2.0 

1.3 

14 

1.3 

2.5 

2.2 

1.1 

1.1 

2.3 

1.1 

1.9 

1.7 

15 

1.8 

2.9 

3.2 

3.8 

1.6 

2.0 

1.2 

1.1 

2.0 

16 

2.6 

3.1 

2.5 

1.3 

1.2 

3.2 

1.3 

1.6 

3.1 

17 

1.8 

3.1 

2.5 

2.1 

1.8 

1.6 

1.3 

1.3 

1.9 

18 

2.7 

1.9 

3.9 

1.4 

1.8 

3.9 

1.3 

1.2 

2.2 

19 

2.2 

2.9 

7.0 

1.4 

1.4 

3.5 

1.4 

1.4 

2.4 

20 

4.2 

2.2 

2.4 

1.8 

1.6 

2.1 

1.1 

1.6 

3.2 

21 

1.9 

2.1 

2.1 

1.4 

1.1 

3.5 

1.4 

1.0 

2.0 

22 

2.5 

2.3 

3.5 

2.5 

0.9 

3.2 

1.6 

1.2 

1.1 

23 

1.9 

1.2 

2.5 

1.4 

1.6 

1.2 

1.2 

2.1 

1.7 

24 

1.9 

1.0 

3.3 

1.4 

1.0 

1.5 

1.1 

1.6 

2.5 

Total 

49.8 

54.3 

83.3 

41.1 

40.1 

70.7 

35.1 

40.0 

63.7 

Mean 

2.1 

2.3 

3.5 

1.7 

1.7 

2.9 

1.5 

1.7 

2.7 

a 

0.7 

0.9 

1.5 

0.7 

1.0 

1.7 

0.5 

0.5 

1.8 
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Table  F.3         Operator  I  -  Tests  5,  6,  and  7 


Test  5 

Test  6 

Test  7 

B 

F 

G 

L 

O 

R 

C 

D 

H 

1 

1.1 

1.2 

5.2 

1.0 

1.8 

3.0 

1.0 

2.5 

6.2 

2 

4.9 

2.0 

4.1 

1.0 

1.1 

3.4 

1.0 

2.2 

2.0 

3 

1.8 

1.9 

1.6 

1.1 

3.5 

2.5 

1.0 

1.7 

2.1 

4 

1.5 

1.4 

2.6 

0.9 

1.6 

2.7 

1.8 

3.1 

2.7 

5 

1.4 

1.4 

2.5 

1.2 

1.3 

1.8 

2.0 

3.2 

4.2 

6 

1.7 

1.7 

2.1 

1.2 

2.3 

7.4 

2.2 

2.9 

1.4 

7 

1.7 

1.6 

1.8 

1.2 

1.4 

2.7 

1.9 

3.6 

1.9 

8 

1.0 

1.0 

2.3 

1.3 

1.4 

3.1 

2.2 

3.0 

5.1 

9 

1.1 

1.0 

4.0 

2.3 

1.4 

2.6 

2.5 

3.0 

1.6 

10 

3.0 

1.9 

3.5 

1.9 

1.7 

1.9 

5.1 

2.2 

1.9 

11 

4.2 

1.8 

1.9 

1.9 

1.8 

2.2 

2.0 

3.5 

4.0 

12 

3.2 

1.4 

2.7 

2.1 

1.6 

6.0 

2.1 

2.9 

1.8 

13 

3.0 

1.8 

6.3 

1.5 

1.6 

3.5 

1.8 

3.9 

4.2 

14 

2.0 

1.6 

3.2 

1.2 

3.5 

1.5 

1.2 

3.0 

3.7 

15 

1.8 

1.0 

2.0 

1.3 

2.1 

1.8 

1.0 

2.2 

2.0 

16 

1.8 

3.1 

1.7 

1.2 

1.3 

1.6 

1.0 

2.0 

3.7 

17 

2.5 

1.4 

2.1 

1.3 

3.6 

1.6 

1.4 

3.3 

1.9 

18 

1.9 

2.7 

4.8 

2.4 

2.2 

3.5 

5.2 

2.0 

3.4 

19 

2.1 

1.3 

3.1 

1.5 

1.4 

1.9 

2.9 

3.6 

2.7 

20 

1.5 

2.2 

2.1 

1.2 

1.4 

4.7 

1.7 

2.6 

1.5 

21 

1.8 

2.1 

1.8 

1.1 

1.8 

2.2 

1.5 

3.4 

2.6 

22 

3.5 

1.4 

2.2 

1.1 

2.3 

1.9 

1.7 

1.5 

4.1 

23 

1.8 

1.2 

1.8 

1.2 

1.3 

2.0 

1.8 

1.0 

2.0 

24 

1.4 

1.0 

1.6 

1.6 

1.0 

1.7 

1.6 

1.1 

3.4 

Total 

51.9 

39.5 

67.1 

33.7 

44.7 

67.1 

47.8 

63.6 

70.2 

Mean 

2.2 

1.6 

2.8 

1.4 

1.9 

2.8 

2.0 

2.7 

2.9 

a 

1.0 

0.5 

1.2 

0.4 

0.7 

1.4 

1.1 

0.8 

1.2 
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Table  F.4  Operator  II  -  Test  1 


60°FOV 

30°  FOV 

15°  FOV 

J 

M 

P 

K 

N 

Q 

L 

O 

R 

1 

1.4 

1.5 

1.9 

2.2 

4.0 

4.4 

2.7 

3.3 

2.6 

2 

1.5 

3.3 

9.3 

1.5 

3.2 

4.9 

2.2 

4.6 

7.5 

3 

1.2 

4.6 

1.5 

4.4 

8.2 

3.6 

5.1 

3.1 

6.5 

4 

1.0 

1.9 

5.5 

2.0 

4.8 

2.7 

1.6 

3.4 

3.0 

5 

1.2 

5.9 

1.2 

2.5 

3.2 

18.3 

3.4 

4.0 

4.2 

6 

1.0 

3.6 

10.4 

1.5 

5.7 

5.4 

6.8 

13.5 

2.1 

7 

3.6 

2.4 

2.1 

7.7 

3.1 

3.7 

7.2 

2.9 

5.0 

8 

6.6 

2.1 

8.7 

1.5 

3.1 

5.6 

8.0 

9.2 

9.1 

9 

1.9 

1.7 

14.3 

2.5 

3.7 

7.9 

3.3 

3.7 

4.6 

10 

2.3 

9.2 

4.4 

1.7 

3.3 

2.5 

3.4 

11.6 

6.6 

11 

1.9 

4.4 

2.6 

12.6 

6.2 

1.7 

3.6 

4.8 

6.1 

12 

2.4 

4.7 

3.8 

4.1 

2.5 

10.0 

3.7 

8.2 

8.1 

13 

3.1 

2.2 

3.8 

3.1 

2.6 

21.0 

2.5 

6.0 

7.0 

14 

6.5 

3.3 

4.4 

5.0 

7.8 

2.9 

4.1 

10.4 

5.5 

15 

1.9 

2.3 

3.4 

2.4 

1.3 

3.9 

3.6 

4.4 

5.2 

16 

2.6 

2.2 

2.1 

8.7 

5.1 

2.1 

2.0 

6.0 

2.9 

17 

1.4 

1.8 

13.5 

8.2 

1.6 

3.3 

6.9 

27.0 

23.5 

18 

8.3 

2.3 

7.4 

2.7 

2.0 

3.3 

1.7 

5.9 

14.1 

19 

4.2 

1.8 

6.0 

4.8 

2.5 

2.2 

3.5 

13.2 

24.5 

20 

2.7 

1.4 

4.9 

3.1 

3.6 

3.7 

3.7 

26.4 

14.1 

21 

1.1 

1.2 

9.6 

4.2 

4.4 

5.4 

28.3 

4.4 

13.4 

22 

1.9 

2.8 

3.7 

3.3 

6.3 

3.0 

3.3 

5.8 

10.7 

23 

3.0 

1.9 

5.1 

2.7 

1.7 

2.9 

9.0 

7.2 

18.9 

24 

9.1 

1.0 

3.8 

4.2 

8.8 

2.0 

13.5 

7.3 

6.6 

Total 

72.1 

69.7 

133.3 

96.9 

99.0 

126.6 

133.4 

196.4 

211.6 

Mean 

3.0 

2.9 

5.6 

4.0 

4.1 

5.3 

5.6 

8.2 

8.8 

a 

2.3 

1.8 

3.6 

2.7 

2.1 

4.7 

5.5 

6.4 

6.1 
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Table  F.5         Operator  II  -  Tests  2,  3,  and  4 


Test  2 

Test  3 

Test  4 

A 

D 

G 

B 

E 

H 

J 

N 

R 

1 

2.0 

1.6 

2.2 

1.2 

2.6 

3.3 

1.2 

2.1 

3.1 

2 

1.2 

1.2 

2.9 

2.2 

2.6 

1.7 

1.2 

1.3 

3.3 

3 

1.4 

2.0 

2.6 

1.4 

5.4 

1.6 

1.0 

1.5 

3.4 

4 

1.3 

5.9 

2.7 

1.4 

3.8 

2.1 

1.5 

2.3 

4.6 

5 

2.5 

3.5 

2.1 

1.6 

2.3 

3.9 

1.4 

1.6 

1.8 

6 

2.0 

2.3 

2.8 

2.2 

1.7 

8.6 

1.2 

2.5 

2.0 

7 

2.1 

2.6 

1.8 

2.2 

2.4 

4.1 

1.1 

2.5 

1.2 

8 

1.4 

1.8 

2.0 

3.4 

2.4 

6.4 

1.0 

1.8 

1.5 

9 

1.6 

2.8 

12.7 

3.4 

2.5 

1.8 

1.1 

1.8 

3.9 

10 

1.9 

2.9 

2.6 

3.6 

1.3 

2.5 

1.0 

1.9 

2.9 

11 

2.2 

3.3 

4.1 

3.0 

1.8 

2.0 

1.0 

1.6 

5.1 

12 

2.6 

2.4 

8.3 

6.0 

1.6 

1.8 

1.3 

1.3 

3.9 

13 

2.0 

13.6 

3.7 

3.6 

1.5 

4.6 

1.8 

1.0 

2.1 

14 

2.6 

3.5 

6.2 

2.8 

8.1 

2.4 

1.5 

1.2 

4.1 

15 

2.0 

2.3 

2.8 

2.4 

3.1 

2.5 

1.4 

1.7 

4.8 

16 

1.8 

2.2 

1.5 

3.0 

3.2 

3.2 

1.8 

4.8 

1.9 

17 

1.3 

2.1 

4.1 

3.4 

3.0 

2.3 

1.7 

1.6 

2.4 

18 

1.5 

3.6 

2.7 

2.8 

3.0 

3.0 

1.7 

2.2 

2.5 

19 

1.0 

2.8 

6.8 

2.0 

4.1 

3.3 

1.8 

1.2 

1.8 

20 

1.0 

2.2 

2.1 

2.3 

2.9 

3.8 

1.8 

1.6 

2.5 

21 

1.1 

1.8 

5.8 

4.1 

4.6 

7.1 

2.0 

1.4 

3.9 

22 

1.8 

2.3 

6.6 

1.9 

4.9 

6.4 

2.5 

1.4 

2.2 

23 

1.8 

1.6 

5.4 

2.0 

2.7 

1.5 

2.8 

0.9 

2.7 

24 

3.7 

2.4 

4.4 

1.7 

3.7 

5.1 

1.8 

1.5 

5.2 

Total 

43.9 

72.6 

98.9 

63.8 

75.3 

85.0 

36.8 

42.7 

72.9 

Mean 

1.8 

3.0 

4.1 

2.7 

3.1 

3.5 

1.5 

1.8 

3.0 

o 

0.6 

2.4 

2.6 

1.0 

1.5 

1.9 

0.5 

0.8 

1.2 
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Table  F.6         Operator  II  -  Tests  5,  6,  and  7 


Test  5 

Test  6 

Test  7 

K 

O 

P 

C 

F 

I 

L 

M 

Q 

1 

1.1 

2.3 

5.2 

1.3 

1.2 

3.2 

2.0 

2.5 

2.1 

2 

1.0 

2.1 

3.9 

2.7 

1.5 

5.3 

1.7 

2.0 

2.8 

3 

1.2 

2.7 

2.8 

3.0 

2.3 

2.8 

1.7 

2.5 

3.8 

4 

1.3 

2.5 

2.6 

2.1 

1.1 

3.7 

5.4 

2.2 

2.8 

5 

1.1 

2.3 

2.5 

1.8 

2.8 

2.9 

2.9 

2.2 

4.5 

6 

1.2 

2.4 

1.4 

2.8 

2.2 

2.4 

2.0 

6.0 

4.0 

7 

1.0 

2.1 

1.5 

1.7 

1.4 

1.8 

2.3 

2.7 

2.9 

8 

1.1 

2.5 

2.1 

1.7 

1.3 

2.1 

4.6 

2.9 

2.3 

9 

1.7 

1.5 

2.2 

2.6 

1.9 

2.3 

3.4 

2.4 

2.9 

10 

2.0 

1.3 

3.0 

2.4 

1.9 

2.0 

4.0 

2.0 

2.4 

11 

2.0 

1.4 

2.7 

2.0 

1.8 

5.2 

2.0 

2.2 

4.0 

12 

2.3 

4.4 

1.8 

2.0 

2.3 

9.5 

3.3 

2.5 

1.5 

13 

2.5 

2.5 

2.7 

1.6 

1.2 

5.1 

2.2 

2.7 

3.1 

14 

2.2 

1.9 

3.5 

2.3 

1.9 

5.1 

1.4 

2.5 

3.0 

15 

2.4 

1.5 

2.2 

2.7 

1.7 

1.4 

2.6 

2.9 

4.9 

16 

2.4 

2.0 

2.3 

1.6 

1.3 

2.5 

5.3 

3.1 

5.1 

17 

1.9 

2.5 

1.8 

1.6 

2.8 

2.8 

2.5 

3.3 

9.5 

18 

1.6 

3.0 

3.0 

1.6 

2.2 

4.1 

2.5 

3.6 

2.8 

19 

2.0 

3.4 

2.2 

1.4 

1.2 

2.4 

2.4 

2.2 

2.3 

20 

1.5 

1.9 

2.1 

1.6 

1.9 

3.0 

2.4 

2.1 

2.4 

21 

2.1 

1.4 

5.5 

1.8 

2.1 

5.3 

2.7 

1.8 

3.5 

22 

5.1 

1.1 

4.6 

1.7 

1.1 

10.4 

1.7 

2.1 

4.3 

23 

2.4 

1.2 

1.9 

2.0 

1.6 

3.8 

2.0 

2.7 

4.3 

24 

1.9 

1.2 

3.1 

2.1 

3.9 

5.5 

4.1 

1.2 

1.7 

Total 

45.0 

51.3 

66.7 

47.9 

44.6 

94.5 

67.2 

62.3 

82.9 

Mean 

1.9 

2.1 

2.8 

2.0 

1.9 

3.9 

2.8 

2.6 

3.5 

a 

0.8 

0.8 

1.1 

0.5 

0.6 

2.2 

1.1 

0.9 

1.6 

122 


Table  F.7         Operator  m  -  Test  1 


60°FOV 

30°  FOV 

15°  FOV 

B 

E 

H 

C 

F 

I 

A 

D 

G 

1 

1.0 

3.0 

2.3 

1.0 

5.6 

3.2 

1.4 

1.0 

23.2 

2 

1.2 

8.1 

1.6 

1.1 

4.2 

5.4 

1.1 

1.0 

41.9 

3 

1.1 

4.2 

4.6 

1.3 

5.8 

6.9 

2.0 

19.4 

3.1 

4 

1.0 

3.1 

2.7 

6.5 

4.9 

10.2 

5.5 

16.5 

29.9 

5 

1.1 

4.0 

2.9 

5.4 

6.5 

4.8 

2.4 

3.4 

4.6 

6 

1.1 

3.6 

1.9 

7.1 

5.1 

4.3 

1.8 

3.1 

3.1 

7 

1.0 

3.0 

2.4 

5.5 

3.5 

12.1 

2.9 

4.8 

3.4 

8 

1.0 

5.6 

2.9 

7.8 

4.4 

17.5 

1.3 

1.9 

4.1 

9 

1.1 

1.4 

1.9 

6.1 

4.7 

15.9 

0.9 

2.9 

7.4 

10 

3.9 

1.1 

2.9 

7.2 

7.0 

9.4 

1.1 

2.7 

10.4 

11 

1.0 

1.1 

1.9 

10.7 

4.5 

14.2 

1.4 

10.9 

3.4 

12 

1.7 

1.3 

1.1 

7.2 

5.1 

6.7 

1.7 

3.0 

4.6 

13 

1.4 

11.1 

2.2 

7.7 

8.7 

4.3 

5.4 

2.6 

7.0 

14 

1.4 

3.4 

3.7 

6.3 

10.4 

5.2 

2.5 

2.2 

5.8 

15 

5.5 

1.7 

1.8 

4.8 

4.7 

5.7 

2.9 

2.3 

2.6 

16 

4.5 

1.2 

5.1 

5.7 

4.4 

2.0 

4.3 

3.0 

3.9 

17 

1.6 

1.8 

1.3 

5.5 

6.5 

2.0 

1.7 

3.3 

1.9 

18 

1.3 

4.3 

4.9 

6.7 

6.5 

13.0 

1.1 

17.8 

5.7 

19 

4.9 

1.7 

12.9 

7.3 

4.1 

1.9 

1.0 

3.6 

15.9 

20 

3.0 

3.0 

6.0 

7.4 

4.6 

1.8 

0.9 

3.7 

29.2 

21 

1.3 

3.1 

4.3 

7.5 

5.4 

2.5 

1.0 

2.2 

14.3 

22 

9.1 

2.8 

4.0 

8.9 

3.0 

1.9 

1.3 

6.4 

2.9 

23 

6.3 

2.0 

4.7 

7.1 

6.3 

3.1 

1.5 

0.9 

2.3 

24 

4.0 

3.9 

12.4 

4.7 

3.2 

7.0 

1.4 

1.0 

4.9 

Total 

60.6 

79.6 

92.4 

146.5 

128.9 

160.9 

48.5 

119.6 

235.4 

Mean 

2.5 

3.3 

3.9 

6.1 

5.4 

6.7 

2.0 

5.0 

9.8 

a 

2.1 

2.3 

3.0 

2.3 

1.6 

4.7 

1.3 

5.3 

10.5 
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Table  F.8         Operator  in  -  Tests  2,  3,  and  4 


Test  2 

Test  3 

Test  4 

K 

N 

Q 

L 

O 

R 

B 

F 

G 

1 

1.4 

2.9 

2.9 

1.0 

1.7 

1.3 

1.9 

1.6 

6.6 

2 

3.1 

1.9 

1.8 

1.0 

1.9 

2.2 

3.7 

1.2 

3.7 

3 

3.4 

3.7 

2.6 

3.3 

2.3 

1.2 

2.7 

1.8 

8.8 

4 

3.4 

1.9 

3.6 

2.8 

3.5 

2.5 

1.2 

4.2 

2.6 

5 

2.0 

1.9 

2.6 

10.5 

1.4 

1.5 

2.1 

4.0 

2.6 

6 

2.9 

2.2 

2.4 

4.5 

4.4 

1.2 

5.8 

2.6 

3.1 

7 

2.0 

1.8 

7.5 

1.2 

1.7 

1.5 

1.2 

1.3 

1.9 

8 

2.0 

3.2 

4.7 

1.8 

2.0 

2.2 

2.2 

1.7 

2.0 

9 

1.4 

2.9 

1.8 

2.2 

7.9 

1.5 

1.7 

2.4 

4.4 

10 

2.1 

1.5 

8.2 

2.4 

6.4 

2.0 

2.8 

1.0 

6.3 

11 

3.9 

1.7 

3.9 

2.1 

2.2 

3.8 

2.2 

1.3 

4.7 

12 

3.1 

2.3 

1.6 

1.6 

1.2 

2.2 

2.2 

1.4 

2.5 

13 

6.9 

1.4 

7.4 

5.9 

3.4 

1.8 

2.5 

1.3 

3.8 

14 

2.7 

2.0 

3.1 

1.0 

1.7 

4.1 

1.4 

1.6 

2.7 

15 

1.6 

2.3 

2.8 

1.0 

1.6 

2.4 

1.5 

1.2 

2.3 

16 

5.3 

3.2 

3.1 

1.1 

4.4 

2.7 

2.1 

1.6 

3.0 

17 

2.5 

2.6 

2.0 

1.2 

2.4 

2.9 

1.7 

2.5 

1.9 

18 

2.0 

2.1 

2.9 

3.4 

6.8 

1.2 

1.6 

2.3 

4.2 

19 

1.9 

2.6 

2.0 

1.6 

3.1 

2.3 

2.7 

1.6 

3.7 

20 

2.1 

2.4 

2.8 

10.4 

1.2 

5.7 

2.3 

1.3 

2.1 

21 

2.3 

1.5 

3.9 

2.2 

3.0 

3.3 

2.5 

1.7 

2.1 

22 

2.9 

1.1 

3.1 

1.7 

2.0 

2.2 

1.9 

1.7 

2.5 

23 

2.6 

1.8 

6.3 

1.0 

1.1 

1.8 

2.6 

1.1 

1.4 

24 

2.2 

1.2 

2.2 

1.0 

1.9 

5.9 

2.2 

1.3 

5.1 

Total 

65.7 

52.3 

85.1 

65.6 

69.3 

59.6 

54.8 

43.9 

84.1 

Mean 

2.7 

2.2 

3.5 

2.7 

2.9 

2.5 

2.3 

1.8 

3.5 

a 

1.2 

0.6 

1.9 

2.6 

1.8 

1.3 

0.9 

0.8 

1.7 
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Table  F.9  Operator  III  -  Tests  5,  6,  and  7 


Test  5 

Test  6 

Test  7 

C 

D 

H 

J 

M 

P 

A 

E 

I 

1 

1.0 

2.1 

1.7 

1.1 

2.6 

1.1 

1.6 

2.5 

2.1 

2 

1.0 

1.0 

1.0 

0.9 

1.5 

3.6 

1.9 

2.4 

6.9 

3 

1.0 

2.1 

2.2 

4.3 

2.3 

2.1 

2.1 

5.5 

6.3 

4 

1.3 

6.3 

2.3 

1.6 

4.0 

2.4 

2.0 

2.5 

4.0 

5 

2.2 

1.5 

2.1 

1.2 

7.2 

1.5 

1.2 

3.5 

4.7 

6 

4.7 

1.5 

2.1 

1.1 

2.5 

4.8 

2.1 

2.9 

2.2 

7 

1.7 

2.2 

1.6 

1.0 

1.9 

2.0 

2.4 

6.3 

4.4 

8 

4.5 

1.7 

3.3 

1.0 

1.7 

4.2 

1.5 

4.6 

3.9 

9 

4.2 

2.3 

1.7 

1.1 

1.4 

2.9 

2.4 

2.6 

2.7 

10 

5.8 

1.2 

3.9 

1.0 

6.0 

4.1 

2.6 

1.9 

2.7 

11 

2.0 

2.4 

3.1 

1.4 

2.6 

4.2 

2.1 

2.0 

7.6 

12 

5.0 

2.3 

1.8 

1.0 

3.8 

3.9 

2.8 

3.1 

6.9 

13 

2.4 

2.4 

2.5 

0.9 

3.4 

4.7 

5.7 

1.9 

4.7 

14 

2.0 

2.4 

1.5 

1.2 

3.3 

6.2 

2.3 

1.8 

1.6 

15 

1.7 

2.6 

1.9 

1.0 

2.9 

3.9 

1.9 

1.7 

1.8 

16 

1.5 

2.0 

2.2 

1.9 

2.6 

2.7 

3.1 

2.0 

3.6 

17 

1.4 

1.9 

2.7 

1.6 

2.2 

3.0 

3.6 

1.3 

3.4 

18 

1.5 

2.4 

2.4 

1.1 

3.4 

2.7 

1.1 

3.2 

7.3 

19 

2.5 

2.2 

2.6 

5.7 

3.6 

3.4 

2.4 

2.1 

3.9 

20 

1.1 

4.1 

2.3 

1.5 

3.8 

3.8 

2.6 

3.8 

2.1 

21 

1.2 

1.1 

4.6 

1.4 

2.6 

4.3 

1.7 

10.6 

4.1 

22 

1.1 

1.0 

5.1 

1.6 

2.4 

7.1 

2.6 

3.6 

1.9 

23 

2.6 

1.0 

2.2 

0.9 

1.9 

2.6 

5.2 

2.9 

2.2 

24 

2.5 

1.0 

1.9 

1.7 

1.7 

1.6 

2.5 

2.9 

2.8 

Total 

56.0 

50.7 

58.6 

37.5 

71.1 

82.8 

59.6 

77.6 

93.7 

Mean 

2.3 

2.1 

2.4 

1.6 

3.0 

3.5 

2.5 

3.2 

3.9 

a 

1.4 

1.1 

0.9 

1.1 

1.3 

1.4 

1.0 

1.9 

1.8 
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Table  F.  1 0        Operator  IV  -  Test  1 


60°  FOV 

30°  FOV 

15°  FOV 

K 

N 

Q 

L 

O 

R 

J 

M 

P 

1 

3.8 

2.4 

2.8 

4.0 

5.3 

8.2 

7.6 

4.7 

3.7 

2 

3.8 

4.8 

3.2 

4.4 

4.8 

8.5 

8.6 

3.1 

5.4 

3 

3.1 

2.9 

4.8 

3.8 

7.6 

5.4 

6.7 

13.8 

5.4 

4 

5.0 

2.7 

2.7 

6.3 

4.3 

7.0 

4.8 

6.6 

11.4 

5 

5.5 

4.1 

3.4 

7.2 

4.1 

8.5 

6.4 

30.9 

5.4 

6 

3.9 

5.2 

2.1 

4.0 

2.6 

2.1 

5.3 

5.2 

29.6 

7 

6.8 

2.7 

2.4 

4.4 

2.7 

6.5 

4.3 

7.6 

5.9 

8 

5.2 

5.0 

3.6 

8.2 

1.8 

4.9 

3.0 

2.5 

36.6 

9 

2.9 

4.3 

2.5 

2.7 

1.5 

19.5 

2.1 

5.6 

8.4 

10 

2.7 

3.6 

2.4 

3.9 

1.4 

5.2 

4.9 

4.5 

17.7 

11 

2.4 

4.9 

2.2 

4.3 

1.1 

17.8 

4.6 

3.2 

11.0 

12 

2.5 

2.6 

2.2 

2.6 

3.0 

18.9 

3.1 

5.5 

8.4 

13 

3.4 

1.5 

3.9 

2.6 

3.8 

12.1 

10.5 

22.0 

12.4 

14 

3.6 

3.2 

3.4 

4.1 

4.4 

16.4 

13.9 

23.6 

21.2 

15 

3.7 

1.9 

3.2 

2.2 

3.8 

7.2 

4.4 

5.0 

5.5 

16 

2.5 

2.2 

3.7 

1.7 

1.4 

8.8 

10.2 

5.0 

6.9 

17 

2.8 

2.4 

2.4 

2.0 

5.1 

6.4 

4.1 

4.9 

5.0 

18 

4.2 

1.8 

4.2 

3.4 

5.1 

6.6 

10.0 

19.8 

4.9 

19 

3.1 

2.2 

1.7 

3.0 

1.5 

2.2 

12.8 

6.5 

7.2 

20 

4.7 

2.3 

8.6 

2.8 

2.5 

7.3 

12.1 

7.4 

4.7 

21 

5.3 

3.6 

2.7 

2.1 

17.3 

9.3 

4.6 

1.4 

5.0 

22 

2.8 

4.7 

3.9 

2.0 

5.4 

7.3 

3.2 

4.1 

6.9 

23 

2.3 

3.2 

6.7 

4.0 

3.5 

11.8 

5.9 

4.2 

12.2 

24 

2.7 

1.4 

4.9 

5.1 

4.0 

2.2 

6.3 

3.0 

8.8 

Total 

88.8 

75.6 

83.4 

90.6 

98.0 

210.3 

159.7 

200.1 

249.6 

Mean 

3.7 

3.2 

3.5 

3.8 

4.1 

8.8 

6.7 

8.3 

10.4 

o 

1.2 

1.2 

1.5 

1.6 

3.2 

4.9 

3.3 

7.6 

8.1 
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Table  F.  1 1        Operator  IV  -  Tests  2,  3,  and  4 


Test  2 

Test  3 

Test  4 

B 

E 

H 

C 

F 

I 

K 

O 

P 

1 

1.1 

1.7 

2.1 

1.1 

1.6 

2.2 

1.2 

1.1 

3.0 

2 

2.0 

2.3 

1.3 

1.2 

4.0 

2.6 

2.3 

1.8 

4.3 

3 

1.6 

2.3 

3.7 

1.0 

2.3 

1.9 

1.8 

2.6 

2.0 

4 

1.7 

2.9 

2.9 

1.2 

1.8 

6.1 

3.6 

2.8 

2.4 

5 

1.7 

3.7 

4.9 

1.8 

1.3 

3.1 

2.1 

2.3 

1.6 

6 

1.4 

2.2 

2.2 

1.5 

1.7 

2.4 

1.6 

1.9 

1.6 

7 

1.3 

1.8 

2.9 

1.3 

1.9 

1.8 

1.7 

1.3 

1.9 

8 

2.3 

2.3 

2.1 

2.3 

1.9 

1.4 

1.1 

3.0 

2.5 

9 

2.1 

2.0 

2.0 

1.9 

1.8 

2.3 

1.2 

1.8 

2.7 

10 

3.8 

2.2 

1.7 

5.9 

1.6 

2.1 

1.4 

1.0 

3.2 

11 

3.9 

2.6 

1.7 

2.0 

1.8 

7.6 

1.6 

4.3 

1.3 

12 

2.0 

1.4 

6.3 

1.8 

3.2 

5.4 

1.8 

1.6 

8.4 

13 

1.4 

1.0 

2.3 

1.4 

1.6 

5.5 

1.4 

1.4 

3.3 

14 

1.4 

1.6 

2.5 

1.1 

2.2 

6.3 

1.2 

1.8 

1.8 

15 

1.2 

1.2 

2.9 

1.3 

3.6 

1.8 

1.5 

1.8 

2.2 

16 

1.3 

1.7 

4.0 

1.4 

2.7 

2.2 

1.8 

1.3 

5.6 

17 

5.0 

1.2 

3.1 

1.2 

1.8 

1.8 

1.8 

1.8 

4.0 

18 

1.5 

2.0 

3.7 

1.4 

1.2 

2.5 

1.5 

1.4 

4.7 

19 

4.1 

1.9 

6.7 

1.3 

1.4 

1.2 

1.7 

1.1 

4.1 

20 

2.1 

1.9 

2.1 

1.6 

1.1 

2.0 

1.6 

1.3 

2.3 

21 

2.0 

2.0 

4.0 

1.1 

1.1 

2.4 

4.7 

1.4 

1.8 

22 

4.3 

1.7 

3.1 

1.3 

1.8 

1.9 

1.9 

1.4 

4.1 

23 

1.4 

1.1 

2.2 

1.1 

1.5 

1.5 

3.4 

1.0 

1.1 

24 

1.0 

1.1 

1.8 

1.0 

1.0 

1.8 

2.6 

1.0 

3.1 

Total 

51.9 

45.9 

72.3 

38.1 

45.9 

69.9 

46.5 

42.4 

73.0 

Mean 

2.2 

1.9 

3.0 

1.6 

1.9 

2.9 

1.9 

1.8 

3.0 

a 

1.1 

0.6 

1.4 

1.0 

0.8 

1.8 

0.8 

0.8 

1.6 
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Table  F.  12       Operator  IV  -  Tests  5,  6,  and  7 


Test  5 

Test  6 

Test  7 

L 

M 

Q 

A 

D 

G 

J 

N 

R 

1 

1.1 

1.7 

1.9 

1.9 

1.5 

2.6 

1.2 

3.3 

1.2 

2 

1.2 

1.1 

1.1 

2.9 

1.7 

2.4 

1.4 

3.5 

3.9 

3 

1.0 

1.8 

1.9 

2.0 

1.4 

2.1 

2.3 

3.0 

7.5 

4 

1.4 

1.4 

1.7 

1.3 

6.5 

2.9 

1.7 

2.6 

2.8 

5 

1.3 

1.1 

1.9 

1.2 

3.1 

1.6 

1.3 

2.3 

1.8 

6 

1.2 

1.1 

1.7 

1.3 

1.2 

1.6 

1.3 

2.4 

4.4 

7 

1.9 

1.4 

6.1 

1.6 

1.4 

1.9 

1.4 

2.6 

2.7 

8 

1.3 

1.1 

1.7 

1.3 

1.4 

2.8 

1.4 

2.6 

3.0 

9 

1.2 

1.3 

1.6 

2.2 

2.1 

2.7 

1.5 

1.4 

3.0 

10 

1.8 

1.5 

4.1 

2.0 

2.5 

4.8 

1.2 

2.0 

2.0 

11 

1.4 

1.9 

1.5 

1.0 

1.9 

2.2 

2.1 

1.6 

4.2 

12 

1.4 

1.8 

1.6 

1.1 

2.0 

3.4 

2.6 

2.0 

4.3 

13 

3.9 

1.4 

1.9 

1.0 

2.7 

4.0 

2.8 

1.2 

3.8 

14 

1.1 

2.4 

2.1 

1.2 

2.4 

8.1 

2.6 

2.0 

10.0 

15 

1.0 

2.0 

11.4 

1.1 

1.4 

3.1 

2.0 

1.4 

2.6 

16 

1.0 

1.6 

2.1 

1.0 

6.2 

2.2 

3.7 

2.4 

2.7 

17 

5.2 

1.5 

3.9 

1.1 

2.6 

4.8 

2.5 

1.7 

2.1 

18 

1.4 

3.0 

5.1 

0.9 

2.4 

4.0 

1.3 

1.8 

2.4 

19 

1.3 

1.6 

2.3 

1.1 

2.9 

5.0 

1.3 

2.0 

1.2 

20 

1.3 

1.7 

3.1 

2.0 

2.2 

1.7 

2.5 

1.8 

2.8 

21 

1.5 

0.9 

3.3 

1.7 

2.1 

1.8 

3.2 

2.3 

3.1 

22 

1.3 

1.0 

2.5 

1.3 

1.7 

2.4 

1.8 

1.3 

2.1 

23 

1.1 

1.0 

1.4 

1.0 

1.2 

1.5 

1.5 

2.0 

2.8 

24 

1.0 

1.0 

2.4 

5.0 

1.1 

3.1 

2.7 

1.4 

1.9 

Total 

37.5 

36.5 

68.3 

38.5 

55.7 

72.9 

47.4 

50.7 

78.2 

Mean 

1.6 

1.5 

2.8 

1.6 

2.3 

3.0 

2.0 

2.1 

3.3 

a 

1.0 

0.5 

2.2 

0.9 

1.3 

1.5 

0.7 

0.6 

1.9 
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APPENDIX  G 


Operator  Learning  Curves 


Estimation  of  the  operators'  learning  curves  involved  four  assumptions. 

•  The  learning  curves  had  positive  slope  at  all  times. 

•  The  learning  curves  were  smooth  functions. 

•  The  learning  curves  were  exponential  functions. 

•  For  a  specified  field  of  view,  operator  learning  was  reflected  primarily  in  the 

standard  deviation  of  the  normalized  course  times  (a  smoothness  measure). 

Each  operator  drove  21  courses  with  a  60°  field  of  view,  three  for  each  of  the  seven 
tests.  The  procedure  below  was  applied  to  the  normalized  data  for  each  operator  to  estimate 
his  60°  FOV  learning  curve. 

•  The  standard  deviations  were  entered  into  a  3  x  7  matrix.  Each  row  represented 

a  specific  window  width  (3  as  discussed  in  section  5.2  (row  1  =  15°,  row  2 
=  30°,  and  row  3  =  60°).  Each  column  corresponded  to  a  given  test  (column 
1  =  test  1,  column  2  =  test  2,  etc.). 

•  The  standard  deviations  for  the  three  window  widths  for  each  test  were  summed 

and  entered  into  a  row  vector  of  seven  elements.  Each  column  continued  to 
represent  a  given  test  as  above.  This  step  yielded  the  summed  standard 
deviations  (SSD). 

•  The  row  vector  elements  were  plotted  with  summed  standard  deviations  on  the 

vertical  axis  versus  test  number  on  the  horizontal  axis. 

•  An  exponential  curve  using  the  method  of  least  squares  was  fitted  to  the  plot  by 

linearizing  the  exponential  form  through  taking  logarithms  [G-l].  This 
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defined  the  operator's  60°  FOV  learning  curve. 

Figures  G.  1  through  G.4  show  the  learning  curves  (graphed  as  lines)  versus  the 
test  number. 
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SSD     4 


0 

Test->    12  3  4  5  6  7 

Figure  G.  1       Operator  1 60°  FOV  learning  curve 


SSD     4 


0 

Test->    12  3  4 

Figure  G.2       Operator  II 60°  FOV  learning  curve 
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SSD     4 


0 

Test->    12  3  4  5  6  7 

Figure  G.3       Operator  EQ  60°  FOV  learning  curve 


SSD     4 


0 

Test->    12  3  4 

Figure  G.4       Operator  IV  60°  FOV  learning  curve 
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